Paul Attewell, are you paying attention?
This “deworm everybody” approach has been driven by a single, hugely influential trial published in 2004 by two economists, Edward Miguel and Michael Kremer. This trial, done in Kenya, found that deworming whole schools improved children’s health, school performance, and school attendance. What’s more, these benefits apparently extended to children in schools several miles away, even when those children didn’t get any deworming tablets (presumably, people assumed, by interrupting worm transmission from one child to the next).
A decade later, in 2013, these two economists did something that very few researchers have ever done. They handed over their entire dataset to independent researchers on the other side of the world, so that their analyses could be checked in public. What happened next has every right to kick through a revolution in science and medicine.
[..]They discovered, first off, that a phenomenal amount of information was simply missing from the original data: For 21% of children, there was no age recorded anywhere (and for more than 10%, no information on their gender). This can happen, sure, but the extent of these gaps wasn’t covered explicitly in the original paper.
When the replication team tried to rerun the original analyses, things got worse. The original paper had 10 tables giving the various results of the trial. Most of these had errors. Some were trivial — eight tables had rounding errors, where 0.745 was incorrectly truncated to 0.74 rather than 0.75, and so on.
[…]In the deworming paper, several findings that were labelled as statistically significant actually were not. One observation — that the pills reduced anemia — was labelled as having a p value of <.05, when in fact it was only 0.194.
This is no small error. And it was just 1 of 11 findings that had their level of statistical significance mislabelled.
Then things get worse. When the replication team began to check the economists’ original code, they found that there were frank errors in the instructions to the statistics package. The wrong commands had been typed into the program, and because of this, the wrong answers had come out.
