Now, a painstaking yearslong effort to reproduce 100 studies published in three leading psychology journals has found that more than half of the findings did not hold up when retested. The analysis was done by research psychologists, many of whom volunteered their time to double-check what they considered important work. Their conclusions, reported Thursday in the journal Science, have confirmed the worst fears of scientists who have long worried that the field needed a strong correction.
Worse, Barrett’s piece ignores empirical evidence. In 2012, Leslie John from Harvard Business School surveyed more than 2,100 psychologists and found worrying levels of so-called “questionable research practices.” More than 40 percent selectively reported studies that “worked.” More than half admitted to checking the statistical significance of their results before deciding whether to collect more data. These practices pollute the scientific literature with false positives and, according to the freely volunteered information that John collected, they are not only common in psychology, but largely accepted—most of her respondents thought they were defensible.