Qdrant Benchmarks: 32K Context Embeddings Still Underperform Chunking in Detail Retrieval

The engineering team at Qdrant used DMIS Lab’s UltraBERT-Embed-150M to investigate a practical question: when an embedding model can process 32K token inputs, is chunking still necessary for retrieval pipelines? They compared single 768-dimensional vectors representing entire documents against 512-token chunked segments across five tasks from the LongEmbed benchmark and 200,000 English Wikipedia articles from MLDR.

Results varied by query type. For tasks requiring precise detail localization—multi-hop QA, meeting summarization, single-sentence retrieval, and large-scale article matching—chunking was superior across the board: on Needle tasks, whole-document vectors achieved only 30% accuracy in identifying the correct article at 32K tokens, whereas 512-token chunks reached 56%; on MLDR, chunking yielded an nDCG@10 of 0.683 compared to 0.509 for whole-document vectors. Whole-document vectors performed slightly better only on SummScreenFD (where queries describe entire TV episodes). Hybrid retrieval, which combines both whole-document and chunk vectors, did not consistently improve outcomes and often performed worse than pure chunking across most tasks.

Qdrant concludes that chunking remains the starting point for detail retrieval, while long-context vectors are better suited for coarse topic-level filtering. 1-bit TurboQuant compression reduces storage costs for 1.43 million chunk vectors from approximately 4 GB to just 140 MB, eliminating storage as a barrier for chunk-based approaches. All data reflects Qdrant’s internal testing, has not been independently reproduced, and covers only English documents; embedding latency was measured on NVIDIA L4 hardware.