Ai2 Releases Benchmark Showing LLMs Are Unreliable at Judging Novelty of AI-Generated Research Ideas
Ai2's new Novelty Bench reveals that LLM-based 'novelty judges' are highly unstable when evaluating AI-generated scientific ideas, with minor prompt changes causing massive accuracy swings.
The Allen Institute for AI (Ai2) released a new study titled "Old Ideas, Novel Problems" alongside its "Novelty Bench" benchmark on October 27, 2026. The research evaluates how reliably large language models (LLMs) can assess the novelty of scientific research ideas, a critical function for AI systems increasingly involved in automating scientific discovery.
Testing six major LLMs, including GPT-5.1, GPT-5.4, and Claude Opus 4.6, the study found extreme instability when judging AI-generated ideas. In one experiment, changing just a few words in the prompt caused GPT-5.4's accuracy to drop from 93.5% to 40.9%—below chance level—on pairs containing one human-authored and one AI-generated idea. This represents a shift of over 50 percentage points due to minor wording variations. By contrast, the same prompt change affected accuracy by only about seven percentage points when judging human-authored ideas, suggesting models are significantly less reliable when evaluating synthetic content.
The findings also indicate that adding retrieval steps or increasing reasoning budgets does not consistently improve judgments. Two purpose-built novelty evaluation systems performed worse than a simple prompted baseline. This highlights a significant limitation in current AI-for-science workflows: the components designed to filter for genuine innovation are currently too fragile to be trusted without substantial human oversight.