ReadingResearchRadarInvestment framework
Sign in / Sign up
← Back to reading
News4 min read·2026-10-21DisconfirmAI Editorial

Quality-Management SaaS Company Octave Reliance's Multi-Tenant AI Architecture: Pushing Isolation into the Runtime, Evaluating Embeddings on Its Own Corpus

When deploying AI on a platform serving over 14,000 customers and hosting more than 2 billion documents, Octave Reliance made two key architectural decisions: pushing tenant isolation into microVM runtime rather than application code, and evaluating embedding models against its own corpus rather than public leaderboards. The latter surfaced a trade-off where 512 dimensions held target accuracy while cutting storage roughly 66 percent against the 1,536 default—a trade-off no public benchmark would have shown.

ImportanceMaterialEvidenceE3 inspectableWrite-upQuick

Original publication time 2026-10-21 01:34 GMT+8

The core asset of a quality-management platform is institutional memory: a similar defect investigated last year, a Corrective and Preventive Action that fixed the root cause, an audit finding that flagged the same gap. That knowledge sits scattered across billions of documents, spanning languages and formats, and no human can search by semantic similarity, so every investigation starts from scratch. The Octave Reliance team saw unlocking this history as its single largest opportunity, and also saw the obstacle: complaint records, audit findings, and corrective actions carry the platform's strictest isolation requirements, and those were exactly the records the AI would need to read. In a platform serving over 14,000 customers across medical devices, pharmaceuticals, and food manufacturing, a single inadvertent cross-tenant disclosure would carry regulatory consequences far outweighing any AI benefit.

Faced with that contradiction, the team made an unusual call: rather than shipping one AI feature, it spent its first cycle building a foundation every future feature would sit on. The reasoning was straightforward—Reliance has over 40 applications, each with different document types and workflows, and building AI for one in a silo meant rebuilding the infrastructure for the next. So they inverted the plan: build the horizontal platform first, then cut vertical feature slices into it. Complaint summarization and triage went first, to prove the architecture before the rest followed.

Sign in / Sign up
ReadingResearchRadarInvestment framework

Four tensions shaped the design, and none could be traded away: making data available without risking inadvertent disclosure, executing agents in isolation, reading content that cannot be trusted, and embedding 2 billion documents without a quarter-long wait. When all four had to hold simultaneously, the option space narrowed considerably.

The first key decision was where to enforce tenant separation. Application code would have worked, but it would have meant every future feature team getting it right on every code path, where one missed check risks inadvertent cross-tenant disclosure. The team pushed isolation into the runtime instead: each user session runs in a dedicated microVM with its own CPU, memory, and filesystem, stopped and sanitized when the session ends. That is hardware-level separation designed to hold regardless of how the agent behaves. AgentCore Gateway authorizes at the tool, operation, and parameter level, and automated reasoning provides formal verification against the defined rules.

The second key decision was how to represent the data. The team evaluated embedding models against its own corpus rather than public leaderboards. The deciding questions were specific to this data: what a photo of a cracked housing has to do with the paragraph describing it, and a German complaint with an English CAPA. Cohere Embed v4's multimodal capability places photos and text in one vector space, and native multilingual support means documents in different languages within one tenant's corpus need no translation preprocessing. But the finding that only a proprietary corpus could surface was the dimension choice: Amazon OpenSearch Service costs scale linearly with dimension count, and Embed v4 supports configurable output dimensions of 256 to 1,536. Testing against representative production queries showed 512 dimensions holding target accuracy while cutting storage roughly 66 percent against the 1,536 default. This trade-off would not appear on any public benchmark, because it depends on the actual document mix of defective-product photos, scanned inspection forms, annotated diagrams, and text.

On results, the Complaint Triage Assistant has been running in production for three months. Octave measures 4 to 6 hours per investigation as the work the pipeline absorbed: extraction across formats, the hunt for precedent, and the reading it takes to tell a genuinely similar defect from a coincidence of wording. The foundation cut deployment time per use case by 50 percent—by Octave's own accounting—and semantic search landed in Q2, two quarters ahead of the original Q4 target.

Boundaries worth preserving: all figures come from Octave's internal testing and production self-reporting, published on the AWS official blog. The latency and cost overhead of the microVM approach is not discussed in the source. The team also drew a line it chose not to cross: the assistant surfaces context and precedent, and the quality handler still makes the call—in an industry where that call carries regulatory weight, that boundary shaped what got automated and what did not. This is one company's architecture experience in a specific regulatory context, not a proof of optimality for runtime isolation across all multi-tenant AI scenarios.

Sources

  • 原始出处aws.amazon.com↗
Continue researching →