Why Adding More AI Agents Made Our System Slower

An asynchronous system built with Python to run multiple LLM agents experienced unexpected latency increases and timeouts as more agents were added. Initial assumptions pointed to slow LLM providers, but the root cause was discovered to be within the application's own code. The investigation revealed that even tiny, seemingly insignificant CPU-bound tasks, such as deserializing responses, accumulate and create a bottleneck when hundreds of I/O operations complete concurrently. This cumulative CPU work blocks the event loop, preventing true parallelism and causing delays. The fundamental solution is not micro-optimization but a change in system architecture to distribute the CPU load across multiple processes instead of relying on a single one.