On 2020-03-21 16:42:23, user Nick wrote:
I attempted to reproduce Tables 2 and 3 (R code included at the end of the post), and obtained these results:<br />
`Table 2, Day 3: reported p=0.005, calculated p=0.0136<br />
Table 2, Day 4: reported p=0.04, calculated p=0.0780<br />
Table 2, Day 5: reported p=0.006, calculated p=0.0148<br />
Table 2, Day 6: reported p=0.001, calculated p=0.0019
Table 3, Day 3: reported p=0.002, calculated p=0.0019<br />
Table 3, Day 4: reported p=0.05, calculated p=0.0429<br />
Table 3, Day 5: reported p=0.002, calculated p=0.0025<br />
Table 3, Day 6: reported p<0.001, calculated p=0.0005`
That is, Table 3 was more or less reproduced, but Table 2 wasn't; most of my p values are around twice the ones in the preprint.
Of the 8 tests, 5 produced warnings because chisq.test() doesn't like cell values of 0 or 1. Using fisher.test() from the "stats" package got rid of the warnings and caused some of the Table 2 p values to move towards the ones in the preprint, but only one (Day 6) got close. It isn't clear to me why one would use Fisher's Exact Test here --- my understanding is that it is not sufficient to invoke per-cell numbers of less than 6, as many authors seem to do, but I don't have a reference to hand for that.
Code:<br />
`t2.d3 <- chisq.test(matrix(c(10, 10, 1, 15), ncol=2))<br />
cat("Table 2, Day 3: reported p=0.005, calculated p=", sprintf("%.4f", t2.d3$p.value), "\n", sep="")
t2.d4 <- chisq.test(matrix(c(12, 8, 4, 12), ncol=2))<br />
cat("Table 2, Day 4: reported p=0.04, calculated p=", sprintf("%.4f", t2.d4$p.value), "\n", sep="")
t2.d5 <- chisq.test(matrix(c(13, 7, 3, 13), ncol=2))<br />
cat("Table 2, Day 5: reported p=0.006, calculated p=", sprintf("%.4f", t2.d5$p.value), "\n", sep="")
t2.d6 <- chisq.test(matrix(c(14, 6, 2, 14), ncol=2))<br />
cat("Table 2, Day 6: reported p=0.001, calculated p=", sprintf("%.4f", t2.d6$p.value), "\n", sep="")
t3.d3 <- chisq.test(matrix(c(1, 15, 5, 9, 5, 1), ncol=3))<br />
cat("Table 3, Day 3: reported p=0.002, calculated p=", sprintf("%.4f", t3.d3$p.value), "\n", sep="")
t3.d4 <- chisq.test(matrix(c(4, 12, 7, 7, 5, 1), ncol=3))<br />
cat("Table 3, Day 4: reported p=0.05, calculated p=", sprintf("%.4f", t3.d4$p.value), "\n", sep="")
t3.d5 <- chisq.test(matrix(c(3, 13, 7, 7, 6, 0), ncol=3))<br />
cat("Table 3, Day 5: reported p=0.002, calculated p=", sprintf("%.4f", t3.d5$p.value), "\n", sep="")
t3.d6 <- chisq.test(matrix(c(2, 14, 8, 6, 6, 0), ncol=3))<br />
cat("Table 3, Day 6: reported p<0.001, calculated p=", sprintf("%.4f", t3.d6$p.value), "\n", sep="")`