If you’ve been reading the articles here about the specificity and sensitivity of testing for COVID-19 you are aware that we lack critical information about the various tests used to diagnose the disease. The New England Journal of Medicine has just published an article about the sensitivity and specificity of testing for the Coronavirus. They should have published such an article four months ago. As the paper False Negative Tests for SARS-CoV-2 Infection — Challenges and Implications is not behind a paywall and is available to everyone, you can click on the link above to read it or download it as a pdf at the end of this piece.
The authors of this article clearly understand the issue of sensitivity and specificity as it relates to the diagnosis of COVID-19. Unfortunately, they have written their three page article in a muddled fashion such that their message is likely to be intelligible only to someone who already understands the problem.
Here’s what I think they’re trying to say, followed by what they should have said, but didn’t. They define sensitivity (the percent of people who have the disease and test positive for it) and specificity (the percent of those without the disease who test negative). They then go on to confirm what I had suspected all along, that we don’t know what is the clinical sensitivity for any of the diagnostic tests for the virus currently in use. The authors mention the importance of Bayes Theorem in understanding the interpretation of test results.
They are focused entirely on the problem of false negative tests to the exclusion of false positives. They say that the sensitivity of the tests available likely is no better than 70%, but then fail to state what such a low sensitivity implies.
Consider a test that is 95% sensitive and 95% specific. First suppose that we test 1 million people who don’t have the disease. Bayes Theorem indicates that 5% will test positive even though no one has the disease, that’s 50,000 false positives. Next imagine that we test 1 million people all of whom have the disease. There should be no negative results, but our test is not 100% specific. At 95% specificity we’ll get 50,000 false negatives.
The results of any test depend on the prevalence of the disease in the population tested. If it’s low we’ll be overwhelmed by false positives. If it’s high false negatives will be the problem. It’s this latter issue that the paper concentrates on. It never addresses the problem of false positives.
If a test if 95% sensitive and specific and the prevalence is 50%, then both its positive and negative predictive values will be 95% – ie, one can be 95% certain that a positive or negative test is a true positive or negative.
There’s nothing in this article that is incorrect. The problem with it is that it’s incomplete. It should have concluded that we don’t know what to make of all the testing for COVID-19. We have tests with low sensitivity, uncertain specificity, and we don’t really know the prevalence of the infection in our population. Thus, we have more of a problem than just that of false negatives. We don’t know what to make of any of the test results. We are recommending that more tests be performed while simultaneously admitting that the more we test, the less certain will be our interpretation of the resulting data. The authors of this paper, given the powerful platform that the NEJM provides, missed a real opportunity to elevate the discussion on how to deal with this pandemic in an intelligent manner.