The Agent Said It Was Done. The Database Disagreed.

AI agents can claim a task is complete, but the underlying database often tells a different, incorrect story. Microsoft's ThinkingBox benchmark addresses this by grading agents not on their words or tool calls, but on the final, verifiable state of backend systems. To truly measure reliability, each business workflow is run 20 independent times, revealing a stark difference between what a model can do once versus what it does every time. While many models perform well on a single attempt, very few demonstrate the consistency to succeed reliably across all trials, highlighting a critical gap in dependability.