How We Reduced LLM Costs by 90% with 5 Lines of Code

An asynchronous Python script for validating LLM prompts was inadvertently generating excessive costs by sending all requests at once, despite being designed to stop after a certain number of successful responses. The issue stemmed from how `asyncio.as_completed` immediately schedules all tasks, leading to a 10x increase in unnecessary API calls. By introducing an `asyncio.Semaphore` to limit the number of concurrent requests, the system was modified to only send requests as needed. This small structural change reduced LLM traffic and costs by 90% without impacting performance, highlighting the importance of efficient asynchronous engineering.