Netflix Engineering Team Explains Core Architectural Decisions for AI Discover: LLM Proposes Paths, Code Decides Execution
A Netflix engineering blog post details the intelligent layer design behind AI Discover. Faced with the contradiction between uncontrollable LLM latency, untrustworthy outputs, and the TV platform's requirement for immediate response, the team chose a plan-and-execute separation model: the LLM can draft request processing paths, but code validates every step before execution, falling back to tested fixed plans upon failure. The article also explains how asynchronous page generation, code-enforced security rules, and model-agnostic design collectively support this architecture.
The Netflix engineering team has publicly disclosed the intelligent layer architecture behind AI Discover in a technical blog post. This feature allows TV users to discover content by selecting themes, with each interaction involving an LLM—generating theme tags, writing recommendation descriptions, and suggesting next steps. However, LLM calls are the slowest and most expensive part of the entire system; tail latencies can spike, and outputs may not match expectations. Meanwhile, TV users expect an immediate response after pressing their remote control; any single failure will be directly displayed on the living room screen.
The core contradiction the team faced is this: traditional recommendation systems rank based on historical behavior and do not need to understand user intent; whereas AI Discover requires an LLM to understand what the user wants and plan how to respond. The simplest implementation would hand the entire request to the LLM, letting it select titles, write tags, and decide the next step. But the Netflix engineering team explicitly rejected this path because 'relying on probabilistic output in scenarios requiring governed results' at scale means rare failures happen multiple times a day.
Their solution is plan-and-execute separation. The system is divided into two parts: a controller and an executor. The controller gathers context and determines a processing plan—either using a verified fixed plan or having the LLM draft a new plan on the fly. Regardless of the source, the plan must pass the same set of validations: whether every step is approved, whether every input has a producer, whether there are loops, and whether it produces exactly one page. After validation passes, the executor runs the steps according to the plan, passing outputs between steps through typed contracts.