Chunk size is query-dependent: a simple multi-scale approach to RAG retrieval

The optimal chunk size for Retrieval-Augmented Generation (RAG) is query-dependent, as different queries require varying levels of detail and context. Committing to a single chunk size during indexing leads to suboptimal performance because no single size is universally best. A proposed multi-scale approach indexes the same corpus at multiple chunk sizes and aggregates the results at retrieval time using Reciprocal Rank Fusion (RRF). This method improves retrieval performance by 1–37% across various benchmarks without needing to retrain embedding models.