ReadingResearchRadarInvestment framework
Sign in / Sign up
← Back to reading
NewsQuick take3 min readDisconfirmAI Editorial

AWS Engineer Chambers: A Single Forward-Pass Decision Model Allows Agents to Skip Unnecessary Memory Retrieval

Mike Chambers, a Senior AI DevEx Engineer at AWS, presented a pattern at AGNTCon + MCPCon North America 2026: using a decider model to perform a lightweight judgment before an agent's memory retrieval. For simple inputs like greetings, the system skips retrieval calls entirely. The decider model returns calibrated probability scores via a single forward pass without generating free-form text, allowing it to be embedded in code branches like an if-statement with language understanding capabilities. Chambers emphasized that this experiment is still in its early stages and developers must verify the reliability of the scores themselves.

2026-11-20 09:37 GMT+8

Say “good morning” to an AI agent, and it might search its long-term memory before saying hello back. Somewhere in that system, a service performs a semantic lookup to determine whether your greeting requires understanding your beverage preferences. Even if you take your coffee very seriously, this seems like a lot of infrastructure for two words.

Mike Chambers, a Senior AI DevEx Engineer at Amazon Web Services (AWS), presented a solution to The New Stack Editor-in-Chief Nick Lucchesi at AGNTCon + MCPCon North America 2026: using a decider model to handle the small judgments that accumulate inside an agent. These models are also known as decision models or System One models; they select from a fixed set of options or rate inputs, returning calibrated probabilities without generating free-form text. This concept was popularized by Jev of TypeSafe AI, and AWS’s Strands Labs released the open-source Strands Decider 2B, a model with two billion parameters, on October 1.

Chambers describes this approach as having an agentic-powered, language-aware if-statement within your code. Developers have used classifiers for years, but what distinguishes this generation of decider models is that general-purpose pretraining allows you to try them on new classification tasks without first training your own model. You provide natural-language context, ask a question, and receive a numerical score that the application uses to choose a branch. In the models Chambers describes, this score is returned in a single forward pass, with no token sequence requiring generation.

Sign in / Sign up
ReadingResearchRadarInvestment framework

This makes decider models suitable for placement on the critical path of every request. Chambers experimented with the open-source Strands framework and Amazon Bedrock AgentCore Memory: he placed Strands Decider before the memory lookup to judge whether an incoming message required fetching information from past conversations. Greetings could pass straight through; requests dependent on what the user said earlier would still trigger a search. He also used a decider after retrieval: while a memory service can return the most relevant matches, some may be unrelated to the current task, so the decider can score each result for relevance before the agent incorporates it into the context.

One decision saves a service call, and another reduces the amount of irrelevant material the model must process. A model router can also use scores to send routine prompts to smaller models and reserve more capable models for complex tasks, saving resources from both the routing decision itself and the downstream work it avoids.

However, Chambers explicitly characterizes his memory experiment as experimental. While a number in a predictable JSON field makes it easy to integrate a decider into an application, determining whether that number is reliable enough to act upon requires testing.

"Will the number be correct? That’s for your testing to figure out," Chambers said. When stakes rise—such as when an agent is about to spend money rather than recommend a coffee—Chambers advocates for setting up checks outside the agent’s loop and requiring human approval for decisions that are difficult to reverse.

Sources

  • thenewstack.ioThe New Stack↗
Continue researching →