HNSW at Scale: Why Your RAG System Gets Worse as the Vector Database Grows

Hierarchical Navigable Small World (HNSW) is a common algorithm in vector databases that can silently degrade retrieval quality in RAG systems as the database size increases. This degradation occurs without errors or latency spikes, causing the context quality for the LLM to deteriorate over time. Using a controlled experiment with the LAION-Aesthetics dataset, the author measures Recall@k to demonstrate how HNSW's performance degrades faster than a flat search with a growing corpus. The analysis highlights the importance of tuning HNSW parameters like `ef_search` to balance retrieval accuracy and system performance in production.