Let Decision Models Watch Your Agents

LLM agents can make critical errors, so a specialized AI "decision model" can act as a real-time monitor to detect anomalous behavior as it unfolds. This monitoring pattern involves continuously feeding the agent's complete execution history, including all tool calls and their results, to the separate decision model. The monitor then assesses whether the agent's actions align with the user's original request and authorized scope, classifying the behavior as normal or anomalous. For instance, such a monitor could successfully flag a purchasing agent that is about to mistakenly authorize a recurring subscription when it was only asked to make a one-time purchase. This allows for a layer of oversight that can catch consequential mistakes before they cause real-world problems.