Human-in-the-Loop Without Killing Throughput

A human-in-the-loop system for a text-to-SQL agent initially required manual approval for all write operations, causing significant delays and "rubber-stamp fatigue" among reviewers. To solve this, a risk-based routing system was developed to automatically score and escalate only high-risk actions. The risk score is calculated using signals like the query's "blast radius," the sensitivity of the target table, semantic novelty, and agreement across multiple generated query variations. Low-risk actions are executed immediately, while high-risk ones are sent to an asynchronous review queue, improving both safety and system throughput.