StateBench: AllenAI Tests How Language Models Track State Changes Across Long Documents

Researchers at AllenAI have released StateBench, a benchmark specifically designed to evaluate whether language models can accurately track how the state of entities, objects, or attributes changes across long documents. Current benchmarks largely focus on factual recall or reasoning over short contexts, leaving a gap in measuring state-tracking fidelity over extended narratives or multi-step processes — a critical capability for agentic systems that must maintain coherent world models. StateBench provides a structured evaluation suite that exposes how well models maintain consistency as state changes accumulate over thousands of tokens. For developers building agents that operate over long documents, codebases, or multi-turn workflows, this benchmark offers a principled way to compare model suitability for state-dependent tasks. The dataset and evaluation code are available on Hugging Face, making it immediately usable for model selection and fine-tuning experiments.