“Where’s Marta?”: How We Removed Uncertainty From AI Reasoning

Large Language Models often struggle with complex reasoning tasks, leading to inconsistent and uncertain results. A hybrid approach overcomes these limitations by combining the generative capabilities of LLMs with the guaranteed accuracy of formal verifiers. This technique uses an LLM to translate ambiguous, natural-language requirements from a logic puzzle into precise formal specifications for a tool called Alloy. By iteratively adding constraints, the formal verifier systematically eliminates impossible scenarios until only the correct solution remains. This human-in-the-loop process builds confidence in the AI's reasoning by providing verifiable, deterministic answers instead of relying on probabilistic generation alone.