A false positive means someone would be told they’d already had coronavirus when they had not — a potential danger as people could then think they were immune to the virus when they’re actually still vulnerable.
“That’s terrible. That’s really terrible,” said Dr. Caryn Bern, one of the authors of the study that looked at the 12 tests.
She said while it’s unrealistic to think all tests will be 100% accurate all the time, their false positive rates should be 5% or lower, or ideally 2% or lower.
“This was a real wake up call for me. We’re not at the point where any of these tests can be used reliably,” added study coauthor Dr. Alexander Marson. “There’s a big danger in relying on them at all, but we hope we get to a point soon where we can rely on these tests.”
The COVID-19 Testing Project is a consortium of researchers and physicians at the University of California San Francisco, the University of California Berkeley, the Chan Zuckerberg Biohub, and the Innovative Genomics Institute.
A rocky road to antibody testing
Bern said one of the reasons for the high false-positive rates is relaxed requirements from the US Food and Drug Administration.
In mid-March, when it became clear that Covid-19 was starting to whirl out of control, the FDA loosened its approval standards in order to get more antibody tests out on the market quickly.
The agency started to allow companies to sell tests without first providing evidence that they worked. Some 175 test developers have taken advantage of these new rules and can legally market their antibody tests without first having their validation data evaluated by the FDA.
“I think we’re paying for this now,” said Bern, a professor of epidemiology and biostatistics at UCSF School of Medicine.
“We have moved quickly and thoughtfully, and we are continuing to learn and adapt based on the real-world experience and data we’re seeing,” Hahn wrote in the statement.
When CNN asked an FDA spokeswoman if there had been any progress with this evaluation effort, she responded that “there isn’t much we can publicly share at this time.”
Federal officials have said that having good antibody testing is critical to getting back to normal, since they might help assess who in the US has immunity.
That’s why the false positives are so dangerous, said Marson, a professor of microbiology and immunology at UCSF School of Medicine.
“False positives will strongly cloud our picture of who has been infected and who has not,” he said. “We need clarity on this basic information to start guiding our path out of the pandemic.”
Relaxed FDA rules
If testing developers want to, they can go above and beyond the FDA’s minimum requirements and have the federal agency review their data showing that the tests work. If the FDA gives a green light, the company is awarded an Emergency Use Authorization.
That test got only one false positive out of 108 blood samples.
The California researchers used blood samples from before July 2018, to make sure they were Covid-free. Each test processed between 99 and 108 samples.
One other test performed better than the Mt. Sinai test on the negative samples, with no false positive results.
Of the other 10 tests, six had accuracy rates around or above 95% for the negative samples and four had rates between 84% and 95%.
These results are based on looking for two types of antibodies. One shows up soon after the infection and the other shows up later.
The researchers also checked whether the tests accurately detected positive samples, but are less confident about those results, because there’s variation in how long it takes people to mount a detectable antibody response.
The California researchers obtained as many tests as possible, but so far have been unable to screen antibody tests by large test makers such as Roche and Abbott, because they require proprietary instruments to run the tests, which the California tests did not have.
They hope to continue screening more antibody tests to see if they work, but warned that even if a reliable test shows you have antibodies, doctors still don’t know what that means. Having antibodies could indicate you’re immune and won’t get infected a second time or it could indicate you have no immunity at all. It could also mean something in between — for example, that you have some limited immunity for a period of a few months.
“We have this huge knowledge gap,” Bern said. “That’s the most important message.”
Dr. Minali Nigam and Arman Azad contributed to this story.