ReadingResearchRadarInvestment framework
Sign in / Sign up
← Back to reading
NewsMaterial4 min readDisconfirmAI Editorial

AWS Solutions Architecture Team: The Core Bottleneck for Multi-Agent System Production Deployment Is Architecture, Not Prompting

The AWS Solutions Architecture team shares production experiences with the Deep Insight multi-agent system, noting that agents do not follow fixed instruction sequences but make autonomous judgments based on context; therefore, prompt engineering alone cannot solve production issues. The article details specific architectural practices including using Graph mode to control execution flow, Agents-as-Tools mode to invoke expert agents on demand, assigning different Claude models by role to balance cost and quality, and isolating AI-generated code in ephemeral containers. It also explicitly acknowledges that the current architecture does not detect prompt injection attacks.

2026-10-22 07:36 GMT+8

Building a multi-agent prototype is now quite easy. Open-source frameworks and cloud services allow teams to run a seemingly impressive demo within days. However, the AWS Solutions Architecture team discovered during the construction of Deep Insight that the real challenges begin when moving from a prototype to a production application. The team’s core judgment is that agents do not follow a fixed sequence of instructions; they make their own judgments based on context. Therefore, prompt engineering alone is insufficient; you must design how the agent thinks, when it verifies, and what it does when errors occur.

Deep Insight is a multi-agent system designed for generating data analysis reports: given a CSV file and a natural language prompt, a group of specialized agents collaborates to complete data loading, analysis, chart generation, calculation verification, and report output. The team divides its architecture into three layers. The first layer is the Request Analysis Layer: the Coordinator routes user requests, the Planner breaks complex tasks into checklists of 5 to 10 steps, and the Plan Reviewer presents the plan to the user for approval or requests modifications. The second layer is the Orchestration Layer: the Supervisor calls the expert agents in the third layer in sequence according to the plan. The third layer is the Execution Layer: the Coder writes analysis code, the Validator re-verifies calculations, the Reporter generates the final report, and the Tracker updates progress status.

Sign in / Sign up
ReadingResearchRadarInvestment framework

The key to controlling execution flow lies in two patterns in the Strands Agents SDK. Graph mode defines the execution order between agents as a directed graph, making manual approval and error handling straightforward through conditional branching—when a user requests a plan modification, the process loops back to the Planner rather than starting over. Agents-as-Tools mode allows the Supervisor to register expert agents like the Coder and Validator as tool calls on demand; each agent works in an independent context and returns only compressed results, thereby controlling context window usage while preserving professional capabilities.

Assigning models by role is another architectural decision rather than a simple model selection. The team uses Claude Haiku for the Coordinator’s fast routing classification to avoid consuming Opus-level costs on simple greetings; it uses Claude Opus combined with Extended Thinking for the Planner’s deep planning; and it uses Claude Sonnet for the Execution Layer agents (Supervisor, Coder, Validator, and Reporter). The Supervisor, Coder, and Reporter enable Prompt Caching for system prompts and tool definitions, which the team states can reduce input token costs by up to 90% on repeated calls; the Coordinator runs only once, while the Validator and Tracker have different contexts each time, so caching is disabled for them. This represents a conscious trade-off between cost and performance.

Security issues faced during production deployment are equally important to architectural design. Python code generated by the Coder cannot be run directly on production servers—the model might delete system files, exhaust resources in an infinite loop, or access other users’ analysis data. Deep Insight’s solution is to create independent ephemeral containers on AWS Fargate for each analysis session, running in a private VPC subnet with no direct internet access, with containers automatically terminating after one hour. ALB sticky session cookies ensure that multiple code executions by the same agent are routed back to the same container.

The team explicitly acknowledges a boundary of the current architecture: container isolation protects infrastructure but does not govern content generated by agents. The Validator re-verifies numbers but does not detect hidden prompt injection instructions in uploaded data, nor does it screen LLM-generated text for harmful content. The team recommends adding Amazon Bedrock Guardrails for content filtering and contextual grounding checks for production deployments, but this is not included in the current open-source version.

LG Electronics’ Korea Sales Division built the ChatInsight system based on the same architecture. The team reported that the original cycle for entire data analysis requests required 2 to 3 weeks (including loss of intent transmission between marketing personnel and data scientists), with specific analysis steps shortened from 3 days to 30 minutes, which they described as a 288-fold productivity improvement. This figure comes from the AWS team’s description of its own customer systems and does not provide an independent benchmark methodology or margin of error.

This is Part 1 of a series, defining five major production challenges and demonstrating how the architecture addresses them layer by layer. Part 2 will discuss specific techniques for Context Engineering, and Part 3 will cover deployment operations from development to production. Deep Insight is open source, offering three deployment methods: local CLI, browser-based Web interface, and AgentCore managed hosting, all sharing the same multi-agent architecture.

Sources

  • aws.amazon.comAmazon Web Services↗
Continue researching →