AI-built baselines quietly erased many science papers' gains
A language model built comparison baselines that erased most of 36 papers' claimed AI gains.
Outside your usual reading: this is a research paper, not a company announcement, but it hands you a question worth asking every time you read about an AI model beating some test. Researchers built a system that automatically generates the simple comparison a new method has to beat before anyone should believe it is actually an improvement. That comparison is called a baseline, the plain method a new approach must outperform to prove it is really better, and the paper's point is that weak baselines have been quietly making ordinary results look like breakthroughs across science.
The team pulled 36 tasks straight out of peer-reviewed papers published in Nature, Science, and the Cell family of journals. Every one of the 36 was a case where scientists had proposed a new deep learning method (an AI model built from many stacked layers that learns patterns from data) to solve a real research problem, the kind of prediction task common in the physical and life sciences. For each task, the researchers had a language model, the general kind of AI that reads and writes plain text, read about the problem and build its own simple baseline from scratch, with no human telling it what that baseline should look like.
The results were blunt. The automatically built baselines recovered a median of 95.5 percent, and an average of 85.7 percent, of the entire gap between a trivial guess (the simplest possible answer, such as always predicting the average result) and the score the published deep learning model actually achieved. In plain terms, most of the improvement scientists were crediting to their new method was, on average, something a simple AI-built comparison could already reach. In 12 of the 36 studies, the simple baseline matched or beat the deep learning model outright. That means in a third of these peer-reviewed papers, the headline result may not represent real progress at all.
Nothing about this is specific to AI hype or journalism. These were peer-reviewed papers in some of the most selective journals in science, vetted by expert reviewers, and the pattern still showed up in a third of them. That is the real finding: checking the baseline is a blind spot even careful review does not catch, because building a serious comparison is extra work nobody is required to do.
The paper states plainly that "there have been numerous cases in which apparent progress has turned out to be an artifact of omitted or missing baselines," and frames the automated check as a practical fix: human peer reviewers cannot keep up with the volume of papers being published, so a tool that asks what the simplest explanation for a result would be can catch inflated claims before they get published.
The transferable habit for him is the same one the paper is really about. Every AI benchmark (a standard test everyone runs) claim he reads, in this edition or anywhere else, implicitly depends on what it was compared against. The headline number is only half the story. The other half, almost always left out, is how strong the thing it beat actually was.
there have been numerous cases in which apparent progress has turned out to be an artifact of omitted or missing baselines.via arXiv.org →