Model Routing Is Simple. Until It Isn’t.

Model routing for AI agents is a complex systems optimization problem, not a simple classification task. Factors beyond model pricing and task difficulty, such as infrastructure caching, system latency, and compliance rules, are critical for effective performance. For instance, caching can make a model with a higher token price cheaper in practice for specific workloads. An approach that optimizes across cost, quality, and latency simultaneously finds a better operating point for the entire system than methods based solely on task difficulty.