ReadingResearchRadarInvestment framework
Sign in / Sign up
← Back to reading
NewsMaterial4 min readDisconfirmAI Editorial

Netflix Engineering Team Explains Core Architectural Decisions for AI Discover: LLM Proposes Paths, Code Decides Execution

A Netflix engineering blog post details the intelligent layer design behind AI Discover. Faced with the contradiction between uncontrollable LLM latency, untrustworthy outputs, and the TV platform's requirement for immediate response, the team chose a plan-and-execute separation model: the LLM can draft request processing paths, but code validates every step before execution, falling back to tested fixed plans upon failure. The article also explains how asynchronous page generation, code-enforced security rules, and model-agnostic design collectively support this architecture.

2026-11-18 01:01 GMT+8

The Netflix engineering team has publicly disclosed the intelligent layer architecture behind AI Discover in a technical blog post. This feature allows TV users to discover content by selecting themes, with each interaction involving an LLM—generating theme tags, writing recommendation descriptions, and suggesting next steps. However, LLM calls are the slowest and most expensive part of the entire system; tail latencies can spike, and outputs may not match expectations. Meanwhile, TV users expect an immediate response after pressing their remote control; any single failure will be directly displayed on the living room screen.

The core contradiction the team faced is this: traditional recommendation systems rank based on historical behavior and do not need to understand user intent; whereas AI Discover requires an LLM to understand what the user wants and plan how to respond. The simplest implementation would hand the entire request to the LLM, letting it select titles, write tags, and decide the next step. But the Netflix engineering team explicitly rejected this path because 'relying on probabilistic output in scenarios requiring governed results' at scale means rare failures happen multiple times a day.

Their solution is plan-and-execute separation. The system is divided into two parts: a controller and an executor. The controller gathers context and determines a processing plan—either using a verified fixed plan or having the LLM draft a new plan on the fly. Regardless of the source, the plan must pass the same set of validations: whether every step is approved, whether every input has a producer, whether there are loops, and whether it produces exactly one page. After validation passes, the executor runs the steps according to the plan, passing outputs between steps through typed contracts.

Sign in / Sign up
ReadingResearchRadarInvestment framework

The key to this model is: the LLM can propose paths, but code decides whether to run them. If the planner responds too slowly or the generated plan fails checks, the system falls back to a tested fixed plan. Users always get a result, though its richness may vary. Netflix refers to this pattern as a plan-and-execute agent, contrasting it with common loop-style Agents (reasoning, calling tools, reading results, reasoning again). Loop-style Agents are flexible but cannot predict in advance how many steps they need or determine if they have gone down a wrong path.

Security rules are similarly enforced through code rather than prompts. Content ratings, maturity restrictions, and content explicitly excluded by users—these cannot rely on the probabilistic compliance of an LLM. The planner sees a request stripped of sensitive information, referencing parameters with placeholders; only after the plan passes validation does the controller fill in real values from the actual request. This also defends against prompt injection: even if the LLM is induced to generate malicious parameters, the placeholder mechanism ensures it cannot directly write actual values.

Asynchronous services are another key decision. Most TV interactions are synchronous: press a button, get a page. But multi-step generation may take several seconds. Netflix changed requests involving generation to be asynchronous: pressing a button immediately returns a confirmation, the TV switches to a 'thinking' state, the executor runs the plan in the background, and notifies the TV to fetch the result when complete. Slow generation no longer slows down other requests, and every button press still receives immediate feedback.

Model-agnostic design allows the system to evolve independently of any single model. All LLMs sit behind a unified interface, and each task can specify different models or capability versions, which can be overridden per-task in real time. Before switching, candidate models are scored on the same tasks by automated checks and LLM judges, ensuring every replacement is a measurable change. Since much of the system's behavior exists in server-side prompts and configuration, swapping models, tools, and capabilities requires no new code or TV app release.

The portability of this architecture depends on one premise: Netflix already possesses mature, composable capability units—search, large-scale ranking, catalog metadata access—which are encapsulated as declarative capability interfaces, where each capability declares its inputs, outputs, visible context, and failure behavior. The article admits that reshaping these systems requires careful work, as Agent orchestration challenges many long-standing assumptions of distributed systems: a capability now must operate in combinations no one designed, used by a caller that chooses to invoke it at runtime. Choosing granularity itself is a difficulty—too fine-grained gives the planner more ways to go wrong; too coarse-grained loses the space to compose richer plans.

For teams without similar infrastructure, the greatest cost of this solution lies not in the intelligent layer itself, but in making existing systems composable. The article does not provide a path from scratch; it describes an incremental transformation by a company that already possesses years of recommendation system accumulation in the era of LLMs. Yet it clearly demonstrates a verifiable architectural principle: when LLM outputs need to enter governed production systems, separate the right to propose from the right to execute, and use code verification rather than prompt constraints to ensure safety boundaries.

Sources

  • netflixtechblog.com↗
Continue researching →