Yet another shoddy paper falls apart under scrutiny. David Lisak’s serial predator theory of campus rape has made him a celebrity. Once a virtually unknown associate professor at the University of Massachusetts-Boston, his work is now cited by White House officials and reporters for major newspapers. His influence is evident in the recent documentary The Hunting Ground, and the producers continue...
More criticism of Goffman’s inner city study
After all I have read about her study, I wouldn’t trust a single thing she writes.
Replication issues hit ethnography: did Goffman make up her book?
On the Run is the story of sociologist Alice Goffman’s six years of immersion in a struggling Philadelphia neighborhood, in which she tells the stories of the “routine stops, searches, raids, and beatings that young men navigate as they come of age.” The book was initially hailed as an ethnographic classic, but to me it raised red flags. Too many of the incidents seemed unlikely, based upon my...
Does Paul Attewell believe in replication?
I believe the answer is clearly “no”, but judge for yourself. The emails make interesting reading. This began when an acquaintance of mine, who does not have tenure and shall remain nameless, asked David Monaghan and Paul Attewell for the statistical code for their paper, “The Community College Route to the Bachelor’s Degree.” The paper was published in a top journal...
How do physicists approach statistical significance?
In particle physics, significance is set at 5 sigma—a p value of 3 × 10–7 or 1 in 3·5 million (if the result is not true, this is the probability that the data would have been as extreme as they are).
Compare that with educational research: we typically set it at 1.96 sigma. Imagine how many studies would never be published if we raised the threshold.
Why most medical research is crap
“A lot of what is published is incorrect.” I’m not allowed to say who made this remark because we were asked to observe Chatham House rules. We were also asked not to take photographs of slides. Those who worked for government agencies pleaded that their comments especially remain unquoted, since the forthcoming UK election meant they were living in “purdah”—a chilling state where severe...
Welcome to the jungle
The New York Times has detailed coverage of the political science gay canvasser scandal. Critics said the intense competition by graduate students to be published in prestigious journals, weak oversight by academic advisers and the rush by journals to publish studies that will attract attention too often led to sloppy and even unethical research methods. The now disputed study was covered by The...
Science retracts study on gay canvassers because it was fake!
This is indeed a very big deal, as Science is the top science journal in the world, and the senior author is a well-respected political scientist. The study was published last December in Science, and received lots of media attention. It found that a 20-minute, one-on-one conversation with a gay political canvasser could steer voters in favor of same-sex marriage. Not only that, but these changed...
Avoid Malcolm Gladwell bearing single studies
Malcolm Gladwell also thought the result was cool. Here is his description his book, David and Goliath: The CRT is really hard. But here’s the strange thing. Do you know the easiest way to raise people’s scores on the test? Make it just a little bit harder. The psychologists Adam Alter and Daniel Oppenheimer tried this a few years ago with a group of undergraduates at Princeton University. First...
“Closing the social class achievement gap” paper
A colleague at NCSU asked my opinion of this paper, which tests the effect of a diversity education intervention on GPA. In general, the paper is quite detailed and the analysis carefully done. But the authors find that their intervention has quite an impact: This main effect was qualified, however, by the predicted interaction of generation and condition [F(1,125)=4.34, p=.039]. Unpacking the...