Can we trust LLMs to judge whether a research idea is novel?

A new study investigates the reliability of large language models (LLMs) in judging the novelty of scientific research ideas. Researchers from Ai2 created a benchmark using peer reviews from ICLR and AI-generated ideas to test models like GPT and Claude. The findings reveal significant instability, where minor changes to prompts can cause accuracy to shift by more than 50 percentage points. Additional context, such as related papers or increased reasoning time, did not consistently improve the models' judgments. This work underscores the current challenges in using AI to reliably automate parts of the scientific discovery process.