A handful of academic publishers are piloting AI tools to do anything from selecting reviewers to checking statistics and summarizing a paper’s findings. In June, software called StatReviewer, which checks that statistics and methods in manuscripts are sound, was adopted by Aries Systems, a peer-review management system owned by Amsterdam-based publishing giant Elsevier. And ScholarOne, a peer...
Gary Pike and Jenny Lee are incompetent and disingenuous
The Review of Higher Ed has taken the unprecedented decision to suspend consideration of manuscripts due to a backlog of accepted manuscripts that need to be published over the next couple of years. The supposed reason? They are having a hard time finding reviewers (see here and here). A simple thought experiment shows why this is false. Suppose the situation with reviewers suddenly grew so...
Sue the journal if it publishes an article critical of your research
Stanford University professor Mark Z. Jacobson has filed a lawsuit, demanding $10 million in damages, against the peer-reviewed scientific journal Proceedings of the National Academy of Sciences (PNAS) and a group of eminent scientists (Clack et al.) for their study showing that Jacobson made improper assumptions in order to claim that he had demonstrated U.S. energy could be provided exclusively...
Evidence of p-hacking in ed policy research and the What Works Clearinghouse
I am really looking forward to this paper: This article presents a pre-analysis plan for analyzing the evidential value in a selection of policy research taken from scholarly journals and two research clearinghouses run by the federal government. The analysis will collect p-values from selected studies and estimate the evidential value that they represent using the newly introduced p-curve. This...
Just how random is the peer review process?
Pretty damn random: The NIPS consistency experiment was an amazing, courageous move by the organizers this year to quantify the randomness in the review process. They split the program committee down the middle, effectively forming two independent program committees. Most submitted papers were assigned to a single side, but 10% of submissions (166) were reviewed by both halves of the committee...
Scamming the peer-review process
In-depth article in Nature about faculty taking advantage of online review systems to get their papers published. They suggest fake reviewers and set up email accounts, so that they end up reviewing their own papers! I love how they were found out: Most journal editors know how much effort it takes to persuade busy researchers to review a paper. That is why the editor of The Journal of Enzyme...