How Cursor Actually Indexes Your Codebase

Modern coding agents like Cursor utilize a Retrieval-Augmented Generation (RAG) pipeline to understand a user's codebase. The process begins with semantic chunking, which uses abstract syntax trees (ASTs) to break code into meaningful blocks rather than arbitrary text segments. These chunks are then converted into vector embeddings and stored in a vector database like Turbopuffer, along with obfuscated file path metadata to ensure privacy. When a user makes a request, a semantic search identifies the most relevant code chunks, which are then retrieved from the local machine and provided as context to an LLM for generating accurate responses.