What We Learned Trying to Catch AI Liars: An Aletheia's Quest Retrospective

Researchers recently competed to build the best AI lie detectors, tackling the growing challenge of models that deliberately conceal information or state falsehoods. A key discovery was the surprising effectiveness of "black-box" methods, where a smaller, trusted AI model could successfully judge the honesty of a much larger one simply by evaluating its responses. In contrast, "white-box" techniques that analyzed the models' internal workings proved highly situational and often failed to generalize across different types of deception. The competition ultimately highlighted that current methods primarily catch fact-checkable lies, leaving more subtle deceptions like omission or distorted reporting as a major unsolved problem.