Malcolm Gladwell also thought the result was cool. Here is his description his book, David and Goliath:The CRT is really hard. But here’s the strange thing. Do you know the easiest way to raise people’s scores on the test? Make it just a little bit harder. The psychologists Adam Alter and Daniel Oppenheimer tried this a few years ago with a group of undergraduates at Princeton University. First they gave the CRT the normal way, and the students averaged 1.9 correct answers out of three. That’s pretty good, though it is well short of the 2.18 that MIT students averaged. Then Alter and Oppenheimer printed out the test questions in a font that was really hard to read … The average score this time around? 2.45. Suddenly, the students were doing much better than their counterparts at MIT.
Turns out that finding can’t be replicated. Here is the key chart; note the huge confidence interval for the original study: