Token Pooling-Aware Training for Multi-Vector Embedding Models

Multi-vector embedding models are highly expressive for search but create a large number of document vectors, leading to significant storage costs. Token pooling reduces these representations by merging similar token vectors, for instance, using hierarchical clustering. While this compression saves space, it can cause a loss of useful information and degrade retrieval quality. A technique called token pooling-aware training teaches the model's encoder to preserve the most important information when vectors are merged. This method results in better retrieval performance for the same storage budget compared to a conventionally trained model.