Improving Best-of-N with Budget-Aware Execution for SWE Agents

Software engineering AI agents can be made more efficient by using budget-aware strategies that adapt to a task's difficulty instead of applying a uniform compute cost. This novel approach evaluates an entire pool of generated solutions, using classifiers and signals like automated test results to predict if the pool is strong enough to stop generating more options. These methods allow for optimizing either cost or speed, ensuring resources are allocated intelligently without compromising the quality of the final outcome.