The Untaught Lessons of RAG Retrieval: Cosine Is Not the Foundation

The standard Retrieval-Augmented Generation (RAG) approach of embedding a query and using cosine similarity is challenged in favor of a new framework. This alternative method treats retrieval as a filtering problem over structured tables derived from documents, rather than a search over free text. It employs three parallel signals: keyword filtering, Table of Contents (TOC) reasoning, and embeddings as an optional final step. This approach separates the precise "anchor" from the surrounding "context" and uses an LLM arbiter to rank hits from all signals, creating a more auditable and effective system.