TurboPuffer CTO and Bridgewater Associates Head of Product Engineering: Permissioned Vector Search for Hundreds of Millions of Private Documents
AI agents need to retrieve both public web pages and enterprise private documents, but each internal file has independent permissions. TurboPuffer CTO Nikhil Benesch and Bridgewater Associates Head of Product Engineering Aaron Linsky explained at the AI Engineer Conference why post-filtering leads to a collapse in recall rates, how pre-filtering degrades into a full scan when users have access to half of the documents, and how they achieved 120ms P90 search latency on 400 million vectors through cluster elimination strategies (vendor-reported data).
When AI agents are granted the ability to retrieve enterprise private documents, the core challenge is not semantic understanding, but permissions. Aaron Linsky, Head of Product Engineering at Bridgewater Associates, introduced at the AI Engineer Conference that Bridgewater's Vault stores approximately 100 million documents, sourced from 135 external sources and internal minutes, with varying permissions for each file. TurboPuffer CTO Nikhil Benesch explained why two intuitive approaches fail: post-filtering retrieves nearest neighbors first and then discards unauthorized documents; if a user can only access a small corner of the vector space, spiral searches consistently fail to gather enough results, causing recall rates to collapse. Pre-filtering filters accessible documents by permission first and then calculates distances one by one; when a user has access to half of the documents, this degrades into a full-scan operation, taking approximately 5 seconds even if all 100 million entries reside in memory. Their solution, native filtering, utilizes vector indexing to cluster similar vectors into groups of about 200, discarding unauthorized clusters before scanning within authorized ones, thereby balancing index acceleration with permission filtering. Benesch stated that this achieves approximately 120ms P90 latency across 400 million vectors (vendor-reported data), though complex permissions such as nested groups require additional handling.