How do physicists approach statistical significance?

In particle physics, significance is set at 5 sigma—a p value of 3 × 10–7 or 1 in 3·5 million (if the result is not true, this is the probability that the data would have been as extreme as they are). Compare that with educational research: we typically set it at 1.96 sigma. Imagine how…

Why most medical research is crap

“A lot of what is published is incorrect.” I’m not allowed to say who made this remark because we were asked to observe Chatham House rules. We were also asked not to take photographs of slides. Those who worked for government agencies pleaded that their comments especially remain unquoted, since the forthcoming UK election meant…

Welcome to the jungle

The New York Times has detailed coverage of the political science gay canvasser scandal. Critics said the intense competition by graduate students to be published in prestigious journals, weak oversight by academic advisers and the rush by journals to publish studies that will attract attention too often led to sloppy and even unethical research methods.…

Science retracts study on gay canvassers because it was fake!

This is indeed a very big deal, as Science is the top science journal in the world, and the senior author is a well-respected political scientist. The study was published last December in Science, and received lots of media attention. It found that a 20-minute, one-on-one conversation with a gay political canvasser could steer voters in favor of…

Avoid Malcolm Gladwell bearing single studies

Malcolm Gladwell also thought the result was cool. Here is his description his book, David and Goliath: The CRT is really hard. But here’s the strange thing. Do you know the easiest way to raise people’s scores on the test? Make it just a little bit harder. The psychologists Adam Alter and Daniel Oppenheimer tried…

“Closing the social class achievement gap” paper

A colleague at NCSU asked my opinion of this paper, which tests the effect of a diversity education intervention on GPA. In general, the paper is quite detailed and the analysis carefully done. But the authors find that their intervention has quite an impact: This main effect was qualified, however, by the predicted interaction of…

Springer withdraws 16 gibberish publications

Luckily, none of them were mine: The publishers Springer and IEEE are removing more than 120 papers from their subscription services after a French researcher discovered that the works were computer-generated nonsense. Over the past two years, computer scientist Cyril Labbé of Joseph Fourier University in Grenoble, France, has catalogued computer-generated papers that made it…

Making my code public

My New Year’s resolution for 2014 is to publicly post all of my Stata code for my projects. Why? This will force me to be more careful with my code, even though I am already somewhat anal when it comes to my code.

Replication of research: more bad news

The amount of evidence demonstrating that most published empirical studies are crap seems to be increasing. See, for example, this article from the Economist. This is a great piece on how a student debunked a psychology study by some big guns in his field. Title: Nick Brown Smelled Bull. Funny, I smell that a lot…