ReadingResearchRadarInvestment framework
Sign in / Sign up
← Back to reading
NewsMaterial3 min readDisconfirmAI Editorial

AWS Explains Agent Prompt Caching Breakpoint Management: Maintaining Hits Amid Tool Changes and Sub-Agent Scenarios

AI Agents must resend full context every round, but prompt caching breaks this cycle. The AWS Machine Learning blog details specific caching patterns for OpenAI models on Amazon Bedrock: how to manage implicit and explicit breakpoints to reuse across rounds, how to append tools mid-session without invalidating existing prefixes, and how sub-agents can share pre-warmed caches. Performance data is based on vendor self-tests.

2026-10-27 01:27 GMT+8

AI agents have no internal memory. Every time they call a tool, read the result, or decide on the next step, they must resend the model their tool definitions, instructions, all previous calls, and results. By turn 20, the model re-reads everything from turns 1 through 19, and you pay for those repeated tokens again and again. The AWS Machine Learning blog uses the movie Groundhog Day to describe this predicament.

Prompt caching saves exactly this redundant computation. When a model reads a prompt, it first performs a write called prefill, processing each input token to build key-value vectors as its working state. Caching stores this state; if subsequent requests start with identical tokens, the service loads the saved state directly, processing only the new additions. Amazon Bedrock offers two modes: implicit, where the service automatically selects the endpoint of reusable prefixes, and explicit, where code marks the endpoint using prompt_cache_breakpoint. These can be used in combination.

The first rule for maintaining hits is prefix stability down to the byte. Tool definitions and instructions should be placed at the beginning, with conversation history and new content appended afterward. Placing a timestamp, username, or request ID inside the cache prefix will invalidate everything that follows. A cacheable prefix must contain at least 1,024 tokens to qualify. The author emphasizes that MCP servers and other tool delivery methods need to be indexed to maintain serialization consistency.

Sign in / Sign up
ReadingResearchRadarInvestment framework

When the tool list changes mid-session, the problem becomes tricky. Adding, removing, or reordering tool definitions invalidates the cached prefix. AWS offers two paths: if known tools are simply invoked in different steps, keep the complete TOOLS list unchanged and use the allowed_tools parameter to restrict the subset the model can actually call; if entirely new tools are needed during the session, append their schemas via additional_tools inputs while keeping the initial TOOLS static. In implicit mode, the cache extends to the new user message (including the new tool definition). However, the additional_tools item itself cannot carry a breakpoint marker.

Sub-agents and side calls are another common scenario. Multiple helpers need to share the same background context but have distinct tasks. The approach is to send a warm-up request to write the shared prefix to the cache, confirm non-zero values for cache_write_tokens or cached_tokens in the usage stats, and then launch helpers in parallel. Each helper reuses the same prompt_cache_key, model, and TOOLS, appending only its own task at the end. If the parent agent requires a summary or evaluation after completion, deep-copying the full history and appending instructions allows the helper to hit the cached historical portion.

In self-tests of GPT-6 Astra workloads over six turns, implicit caching reduced input costs by approximately 62% compared to no caching, while explicit rolling breakpoints reduced costs by about 61%. In another scenario involving an 8,000-token startup prefix plus six turns of 2,000 tokens each, static caching lowered costs to $0.50, and rolling breakpoints reduced modeled input costs from $0.87 to $0.32, a 64% decrease. The authors explicitly note these are results for specific workloads and advise readers to measure savings for their own agents. Cache writes are billed at 1.25 times the normal input price, while read prices depend on the model.

For production monitoring, the Amazon Bedrock Runtime endpoint provides CloudWatch metrics for CacheReadInputTokenCount and CacheWriteInputTokenCount; the Mantle endpoint currently lacks independent cache counters, requiring application logs to record response usage fields. Cached prefixes have a minimum lifespan of 30 minutes, refreshed upon successful reuse. Reuse is restricted to the same model, AWS account, and region. The authors finally warn that using caching carries risks of tool and system prompt leakage, which should be included in responsible AI governance.

Sources

  • aws.amazon.comAmazon Web Services↗
Continue researching →