StateBench: Testing how language models track state changes over long documents

StateBench is a new benchmark designed to evaluate how well language models track state changes over long documents. It defines four levels of difficulty, from simple immutable assignments to complex, interdependent updates that require following a chain of changes. Experiments were conducted on open pretrained models and frontier models like Gemini 3.8 Flash, Claude Opus 5.5, and GPT-6. While most models performed well on simpler state-tracking tasks, their accuracy dropped sharply on harder levels as the input length increased, highlighting current architectural limitations in handling long-range dependencies.