Vector Database Company Qdrant: Asymmetric Quantization Reduces Single-Vector PDF Retrieval Memory to 64 Bytes per Page, Retaining 99.2% Quality
Qdrant blog author Evgeniya Sukhodolskaya tested the single dense vector retrieval capability of Cohere Embed 5 Pro across five public English corpora in ViDoRe V3, finding its ranking quality exceeded that of several late-interaction models. The team further stacked Matryoshka dimension truncation, 1-bit document quantization, and 8-bit asymmetric query quantization, combined with top-20 full-vector re-ranking, achieving 99.2% retained ranking quality at a RAM cost of only 64 bytes per page.
When searching PDF page images, the Qdrant team previously typically recommended using late-interaction visual models (such as ColPali or ColQwen), retaining one vector for each image patch to avoid losing relationships between words in the layout during OCR extraction. However, late-interaction methods generate approximately one thousand vectors per page, and their MaxSim similarity metric cannot directly utilize HNSW indexing, making scaling difficult on datasets with billions of points. Out of curiosity, the team tested Cohere Embed 5 Pro—a model that represents an entire PDF page image with a single dense vector—on five public English corpora from ViDoRe V3 (computer science textbooks, financial reports, EU labor reports, US Air Force technical manuals, and US FDA pharmaceutical documents, comprising 12,969 page images and 1,489 English questions). The results showed that its nDCG@10 score surpassed that of several late-interaction models.
The team then explored how to turn this finding into a scalable solution for production environments. Embed 5 outputs default 2048-dimensional vectors, which consume 8 KB per page when stored as float32. Qdrant’s approach involves stacking three compression techniques: first, leveraging Matryoshka Representation Learning (MRL) supported by Embed 5 to retain only the first 512 dimensions of the vector rather than all 2048; second, applying 1-bit binary quantization to document vectors, keeping only the sign (whether each dimension is greater than zero); and crucially, asymmetric quantization—where query vectors maintain 8-bit precision while document vectors use only 1 bit. The intuition is that when dimensions are truncated to 512, the equal-weight sign count of 1-bit quantization becomes coarse, giving the same weight to a dimension just barely above zero and one with a large positive value; retaining magnitude information on the query side allows important dimensions to carry more weight during scoring. During retrieval, compressed vectors are used to find the top 20 candidates, which are then re-ranked using the complete float32 vectors stored on disk.