Optimizing Vector Search: Why You Should Flatten Structured Data

Embedding raw structured data like JSON directly into vector databases for RAG systems results in poor performance. This is because common embedding models are trained on unstructured text and are not optimized for JSON's syntax, which introduces noise through characters like braces and quotes during tokenization. A more effective approach is to "flatten" the JSON, converting its key-value pairs into a natural language sentence or paragraph. An experiment using the Amazon ESCI dataset showed that this flattening method improved retrieval metrics like precision and recall by nearly 20% compared to embedding raw JSON.