The Open Agent Leaderboard

The Open Agent Leaderboard has been launched to evaluate and compare full AI agent systems, not just the underlying models. It measures both performance quality and operational cost across six diverse benchmarks to assess an agent's generality, or its ability to work in unfamiliar settings. The evaluation framework uses a unified protocol so different agents can be tested on benchmarks for coding, web research, and customer service. Early findings indicate that the agent's architecture is as critical as the model itself, and that general-purpose agents are already competitive with specialized ones.