Stop Ranking Agent Configs by Average Score

Relying on average scores to evaluate AI agents is a flawed method that overlooks how components like the model, prompt, and tools interact. A superior approach is Best-Worst Scaling, where a judge selects only the best and worst outputs from a batch, allowing a model to calculate utility scores that account for competition strength. This method uniquely reveals crucial interaction effects, providing clear evidence for which complete agent configurations to deploy, prune