On 2020-05-01 10:16:00, user mendel wrote:
The specificity trials on page 19 are not normal.<br />
7 trials show 100%, with N=30,70,1102,300,311,500,99, sum 2412.<br />
The 6 remaining trials:
368/371 = 99.2% (97.7-99.8)<br />
198/200 = 99.0% (96.4-99.9)<br />
29/31 = 93.6% (78.6-99.2)<br />
146/150 = 97.3% (93.3-99.3)<br />
105/108 = 97.2% (92.1-99.4)<br />
50/52 = 96.2% (86.8-99.5)
Pooling these, I get 896/912=98.3% (97.2-99.0).
"We use the pooled test performance based on the available information:<br />
Sensitivity: 82.8% (exact binomial 95CI 76.0-88.4%)<br />
Specificity: 99.5% (exact binomial 95CI 99.2-99.7%)"
There is no trial that has exactly 1 false positive. There are 3 <br />
trials that don’t have 99.5% in the 95% CI (4 trials if you include <br />
1102/1102). There is no trial that falls inside the 99.2-99.7 range (one<br />
straddles it). The specifity range they’re using is an empty space <br />
between the values that the trials are actually at. This is not a normal distribution.
187 samples had loss of smell and taste in <br />
the past 2 months. This is a very specific indicator for Covid-19, ~70% <br />
of patients (well, 33,9–85,6%, depending on the study, e.g. <br />
Mons/Belgium, Heinsberg/Germany) have that, and I don’t think this kind <br />
of nerve affliction has been reported for any other common illness. Yet <br />
only 11% of these samples test positive. For the 59 more recent samples,<br />
it’s 22%.
This looks like the prevalence this study should have measured is <br />
267/3330 = 8%, and the test failed to pick up on that. It would fail to <br />
pick up on recent infections, because they wouldn’t have seroconverted <br />
(created enough antibodies) yet, and it would fail to pick up on <br />
infections that happened too long ago (because the antibody levels would<br />
have fallen off below the sensitivity of this test). This study really <br />
needed a more sensitive test, like an ELISA, which is actually available<br />
at Standford, and is able to detect much lower levels of antibodies.
This kit has not been validated against people who had the infection a month ago.
The presence of false positives is an indication that cross-reativity <br />
with outher cold viruses exists. If you test a sample with few people <br />
who haven’t had a severe cold recently, which probably includes most <br />
samples taken of people who check into the hospital for elective <br />
surgery, or samples taken in the summer months, you get an optmistic <br />
sensitivity that does not apply to the general population in early <br />
spring.
The WHO Early investigation protocol (Unity protocol) for the <br />
investigation of population prevalence mandates the use of an ELISA <br />
test, or the freezing of samples until a time when such a test becomes <br />
available. The WHO does not endorse the use of lateral flow assays for <br />
this kind of testing.<br />
—-<br />
P.S.: No study that does not measure prevalence in the older <br />
population where the majority of deaths occurs should speak on fatality <br />
rates. This study had 2/167 positives in the age 65+ population, that’s <br />
0.1-4.3% (95% CI), a 30-fold spread, and hardly a representative sample,<br />
since I don’t expect residents from care homes were able to attend the <br />
drive-through testing.