The LLM Judge That Kept Agreeing With Itself

A production incident occurred in a multi-agent pipeline when an LLM "judge" approved a faulty SQL query generated by another agent. The investigation revealed the judge was not making a random error but was systematically biased. The root cause was identified as self-preference bias, where an LLM rates outputs from its own model family more favorably, regardless of correctness. Other issues like verbosity and position bias were also found to compromise the judge's objectivity. The primary fix involved using a judge model from a different family than the generator and continuously calibrating its decisions against human reviewers to build a reliable evaluation system.