Ioannidis is a fascinating guy: years ago he showed that half of all medical research is wrong. You can read more about him in this Atlantic profile; here is his latest: Nevertheless, despite the cynical realization that the methodological norms of science had been neglected (or perhaps because of this realization), voices struggling for more communalism, universalism, disinterestedness, and...
Professor banned from campus because he said his colleagues publish in predatory journals!
Which bring us to the curious and disturbing case of Derek Pyne, an economics professor at Thompson Rivers University (TRU) in Kamloops, B.C. Prof. Pyne was banned from campus and suspended, with pay, by his school this past summer over research he conducted into so-called predatory journals – publications that offer academics a home to publish their research (sometimes for as little as a hundred...
HARKing: Hypothesizing after the results are known
This article considers a practice in scientific communication termed HARKing (Hypothesizing After the Results are Known). HARKing is defined as presenting a post hoc hypothesis (i.e., one based on or informed by one’s results) in one’s research report as if it were, in fact, an a priori hypotheses. Several forms of HARKing are identified and survey data are presented that suggests...
The problem with quant research that most quant folks refuse to acknowledge
This is one of the reasons I am a big proponent of replication and transparency in what we do. How is it that economists, working in good faith, wind up with dubious results? To start, they can overanalyze the data. Modern computers spit out statistical regressions so fast that researchers can fit some conclusion around whatever figures they happen to have. “When you run lots of regressions...
Remove data labels and change data values to reduce bias in research!
Decades ago, physicists including Richard Feynman noticed something worrying. New estimates of basic physical constants were often closer to published values than would be expected given standard errors of measurement1. They realized that researchers were more likely to ‘confirm’ past results than refute them — results that did not conform to their expectation were more often...
One-study wonders, research, and the minimum wage
Megan McArdle provides a nice overview of what we know about minimum wage hikes. What really struck me was her comments on “one-study wonders”: Alas, if only economic analysis were so easy. In fact, it’s very hard to study what happens when we raise the minimum wage. The people confidently proclaiming their ability to see the future are often what I like to call “one-study...
Who is exaggerating research findings in the media?
Interesting question: is it academic researchers overselling their findings, or journalists hyping their story? The researchers found that 33 to 40 percent of the academic press releases exaggerated causation, advice, or inference in some way, and that the news articles took those claims and published them as is, or, in some cases, exaggerated them even more. Now, to be fair, 10 to 18 percent of...