Loop Engineering for RAG Generation: Iterate top-k One at a Time

Retrieval-Augmented Generation (RAG) systems can optimize how they feed retrieved documents to a large language model for answer generation. A common "batch" method sends all top-K retrieved candidates at once, which can be inefficient for simple questions where the answer is in the first result. An alternative "sequential" method processes candidates one-by-one, using a sufficiency signal from the model to stop as soon as a complete answer is found, thereby saving significant token costs. The optimal approach is to use a dispatcher that analyzes the parsed question type, routing listing or comparison questions to the batch method and simple factual lookups to the more efficient sequential method.