Agent Observability Powers Agent Evaluation

Building reliable AI agents requires a shift from debugging code to debugging the agent's non-deterministic reasoning process, which only emerges at runtime. Traditional software observability and evaluation methods fail because the source of truth moves from static code to dynamic execution traces that show what an agent actually did. New observability primitives are needed, such as runs for single steps, traces for complete executions, and threads for multi-turn conversations. These detailed traces directly power agent evaluation, enabling testing at various levels of granularity from individual decisions to entire conversational flows. Consequently, production data becomes a crucial source for discovering failure modes and creating relevant offline tests.