On 2021-06-23 21:55:50, user David Wiseman PhD wrote:
Summary:<br />
Regarding the continued and unnecessary confusion related to the Argoaic and Artuli comments.<br />
1. These are in reality distractions from the central issue that the original NEJM paper remains uncorrected in NEJM as to shipping times. Although a secondary issue, also uncorrected is the "days" nomenclature that is the reason for confusion in the Argoaic and Artuli comments on this forum. Also uncorrected in the original paper is the exposure risk definition which were informed were also incorrect. Together, these issues controvert the conclusions of the original study.<br />
2. The incorrect nomenclature for "days" in the NEJM paper as well as in a follow up work (Clin Infect Dis, Nicol et al.) inflates the number of "elapsed time" days. This has not been corrected by the original authors. We on the other hand have corrected this by providing the correct information in our preprint.<br />
3. Dr. Argoaic seems to have been given a wrong and earlier version (10/26) of the data which, although contains a variable that is supposed to correct the above problem, does not. In fact one cannot come to any conclusion that there is a discrepancy based on this incorrect 10/26 version, unless you have some preconceived notion.<br />
4. Other post hoc analyses reported in follow up works (including social media) by the original authors looking at time from last exposure, or using a pooled placebo group, although flawed for a several reasons, when examined closely, nonetheless support our conclusions that early PEP prophylaxis with HCQ is associated with a reduction of C19.
Detail:<br />
Any confusion about "days" would disappear once the original authors correct the NEJM June 2020 paper as well as a follow up letter in Dec 2020 Clin Infect Dis (see upper red graph in Nicol et al. pubmed.ncbi.nlm.nih.gov/332... "pubmed.ncbi.nlm.nih.gov/33274360/)"). These errors inflate the "DAYS" by 1 day because the nomenclature for describing "days" was incorrect. As far as we know those corrections have not been made in the journals where these errors appear and in a way that can be retrieved in pubmed etc..
As far as we can tell, anyone who has cited the NEJM paper (NIH guidelines, NEJM editorial, many meta-anlayses etc., our protocol in preprint version) also misunderstood the "days" to mean the inflated figure. So the authors need to correct this. As far as we know we are the only ones to do this. After we were informed of this error by the PI (who was unaware of the problem himself) we described this problem very clearly in our preprint, distinguishing between elapsed time and the day on which a study event occurred. For the benefit of those who remain confused, we will endeavor to make it even clearer in a future version. You can read our correspondence log referenced in the preprint to verify that the incorrect "days" nomenclature was unknown to the PI, at least until 10/27 when he informed us about it.
You are confusing "DAY ON which an event occurred" with "DAYS FROM when an event occurred." For example the original NEJM Table 1 says "1 day, 2 days etc." for "Time from exposure to enrollment". This falsely inflates the number of elapsed time days by 1, and as the authors informed us (documented in our preprint), this really means DAY ON which enrollment occurred, with Day 1 = day of exposure, so you need to subtract 1 from the days to get elapsed time FROM exposure. The same error is repeated in Nicol et al. (note: we discuss other unrelated issues relating to time estimates in our preprint).
To confuse matters further, the problem is not even corrected in the dataset linked (datestamp 10/26/20) in the Argoaic comment. In column FS there is a variable "exposure_days_to_drugstart." This appears to indicate elapsed time (ie DAYS FROM) when it actually means the "DAY ON" nomenclature. We were only informed of the nomenclature error on 10/27/20 and later provided with a new version of the dataset on 10/30 where an additional variable "Exposure_to_DrugStart" (column GR) was provided that corrects this error by subtracting 1 from all the values.
Why the Argoaic comment does not link to the correct 10/30 version is unclear, but in this incorrect 10/26 version, the values for the new variable "Exposure_to_DrugStart" (column GR) are IDENTICAL to those in the "exposure_days_to_drugstart" (column FS) variable (they should be smaller by 1). Accordingly, unless Drs. Argoaic and Artuli had a preconceived notion (without checking the data) that some alteration had occurred, it is impossible to draw such a conclusion (albeit one that is incorrect for other reasons) from this incorrect 10/26 dataset. A number of colleagues have downloaded the 10/26 dataset from the link provided in the Agoraic comment, and have verified this problem.
So in addition to the original data set released in August 2020, as well as the three revisions (9/9, 10/6 and 10/30) we describe in our preprint there is this incorrect 10/26 version. I don't know how many people this affects but it would be appropriate for them to be notified that the version they have may be an incorrect one. An announcement on the dataset signup page covidpep.umn.edu/data would also be in order (nothing there today).
Regarding the possibly higher placebo rate of C19 on numbered day 4 (18.9%). This is matched by a commensurate change in its respective treatment arm, yielding RR=0.624 similar to that for numbered days 2 (0.578) and 3 (0.624), justifying pooling. We don't know if the 18.9% represents normal variation or has biological meaning.
Although they used enrollment time data (completely irrelevant to considering whether or not early prophylaxis is beneficial), the original authors (Nicol et al.) in a post hoc analysis, used a pooled placebo cohort to compare daily event rates (red bar graph). This would mitigate possible effects of an outlying value in the placebo cohort. We applied this same pooled placebo method to the data that correctly takes into account shipping times. This method is still limited because it may obscure a poorly understood relationship between time and development of Covid-19. Although at best this would be considered a sensitivity analysis, we did it to answer the Artuli question. This approach yields the same trends as our primary analysis. Using 1-3 days elapsed time of intervention lag (numbered days 2-4) for Early prophylaxis, there is a 33% reduction trend in Covid-19 associated with HCQ (RR 0.67 p=0.12). Taking only 1-2 days elapsed time intervention lag, we obtain a 43% reduction trend (RR 0.57 p=0.09). This analysis appears to reveal a strong regression line (p=0.033) of Covid-19 reduction and intervention lag.
We also looked at the post hoc analysis provided by the original authors (Nicol et al.) that used “Days from Last Exposure to Study Drug Start,” a variable not previously described in the publication, protocol or dataset, so we have no way of verifying it from the raw data. As in a similar PEP study (Barnabas et al. Ann Int Med) this variable has limited (or no) value, as we are trying to treat as quickly as possible from highest risk exposure, not an event (ie Last Exposure) that occurs at an undefined time later. (even the use of highest risk exposure has some limitation, which the authors pointed out to us and which we discuss in our preprint). Further the Nicol analysis used a modified ITT cohort, rather than the originally reported ITT cohort. with these limitations, pooling data for days 1-3 and comparing with the pooled placebo cohort (yields a trend reduction in C19 associated with HCQ (it is unclear which "days" nomenclature is used) after last exposure from 15.2% to 11.2% (RR 0.74, p=0.179).
Taken together with these "sensitivity" analyses inspired by the original authors' methodology, suggests that this is not an artifact of subgroup analysis. It could be said that any conclusions made by the sort of analyses conducted by Nicol are equally prone to the "subgroup artifact" problem. (also note that in our paper, the demographics for placebo and treatment arms in the early cohort match well).
Mention has been made elsewhere of two other PEP studies (Mitja, Barnabas) which concluded no effect of HCQ. It is important to note that the doses used in these studies were much lower than those used in the Boulware et al. NEJM study. Further, according to the PK modelling of the Boulware group (Al-Kofahi et al.) these doses would not have been expected to be efficacious (the Barnabas study used no substantial loading dose). So citing the Mitja and Barnabas studies to support claims of HCQ inefficacy in the Boulware et al paper is unjustified. On the contrary, taken together three studies suggest a dose-response effect. We discuss this in detail in our preprint.
Lastly it is important to note the since the original NEJM study was terminated early, the entire original analysis can be thought of as a subgroup analysis, with all of the attendant problems referenced by the original authors (and us). There is certainly a great deal of under powering and propensity to Type 2 errors, among the issues inherent in a pragmatic study design. The study was not powered as an equivalence study and so no definitive statement can be made that the HCQ is not efficacious. Along with the still uncorrected (in the original journal) issues of shipping times, "days" nomenclature and exposure risk definitions, there are are certainly many efficacy signals that oppugn the original study conclusions,and controvert the statement made in a UMN press release (covidpep.umn.edu/updates) "covidpep.umn.edu/updates)") that the study provided a "conclusive" answer as to the efficacy of HCQ.
_________________<br />
Please note that despite our offer to Dr. Argoaic to contact us directly to walk though the data to try to identify any issues, we have not been contacted.That offer is still extended to anyone who remains confused. We have also attempted to locate both Drs. Argoaic and Artuli to try to clear up their confusion, but these names do not exist in the mainstream literature (i.e pubmed, medrxiv), nor do they appear to have any kind of internet footprint.
With regard to Table 1 of our preprint, the reason why there are no patients for “Day 1” is that there were no patients who received drug the same day as their high-risk exposure. This is consistent with the PIs comment on 8/25/20 (p10 of email log) (at a time when he thought that there was a “Day zero”) “Exposure time was a calculated variable based date of screening survey vs. data of high risk exposure. Same day would be zero. (Based on test turnaround time, I don’t think anyone was zero days).”
We notice an obvious typo in the heading for the second column of our Table 1, which says “To”. But it should say “nPos”, to match the 5th column (and other tables). It is patently absurd that there should be a category of “1 to 0” days or “7 to 5” days etc. “From” makes no sense either and these typos have absolutely no effect on the analysis, interpretation or conclusions. This will be corrected in a later version.