We Taught AI to Answer Questions. Who Taught It to Change Its Mind?

An experiment tests how well AI systems update their conclusions when underlying facts change. Using a custom benchmark called RevisionTrace, six different update strategies were evaluated by comparing their performance when calculating from scratch versus revising a pre-existing state. While all strategies achieved 100% accuracy from a fresh start, their performance dropped dramatically when updating, with one strategy falling to 0.7% accuracy. The analysis identified common failure modes like retaining stale data and causing collateral damage to unrelated facts. A proposed solution using dependency tracking proved to be both 100% accurate and more efficient than a full recomputation.