Lecture

DeepSeek released its AI model DeepSeek-V4.1-Flash-1017 on October 17, 2026, making the model weights available for free on Hugging Face under the MIT license on the same day, permitting commercial use, modification, and redistribution. The model is a minor update to V4.1-Flash released in September, supporting text and image inputs with a context length of 1 million tokens.

According to benchmark results published by DeepSeek, the new model outperformed Anthropic's top-tier paid model Claude Opus 5 in four metrics: scoring 77.0 on the code repair benchmark DeepSWE v1.1 (vs 74.0 for Opus 5), 60.5 on the business automation benchmark Automation-Bench v1.0.6 (vs 50.3), 35.4 on Agents' Last Exam (vs 28.6), and 72.1 on Apex Bench v1.1 (vs 65.8). These results are vendor self-reported and have not been independently reproduced.

Despite strong performance in some tasks, the model scored 42.3 on Terminal-Bench 4.0, which involves terminal-based coding, still trailing Opus 5's 51.8. It also significantly lagged behind Opus 5 on the difficult exam HLE, scoring 42.6 against 56.3. The model uses a Mixture-of-Experts (MoE) architecture with 552B total parameters, activating 8B during input and 16B during output, and includes a 196B parameter conditional memory module called Engram.

On October 17, 2026, DeepSeek announced in its API update log that it is opening its in-house search engine to external developers. Previously serving only DeepSeek's own web interface and chat API, the search capability is now available as independent "Search" and "Fetch" tools.

According to official documentation, the DeepSeek search engine is an AI-native general-purpose search engine that indexes hundreds of billions of web pages globally, supports multilingual search, and returns highly relevant and authoritative results in real time. It employs strict detection mechanisms to filter out SEO spam, GEO manipulation, and low-quality AIGC content, ensuring trustworthy information for users and LLMs.

The release includes two main interfaces: the Search API finds web pages for a query and returns structured results, while the Fetch API grabs a single page and returns it as Markdown. This allows developers building AI agents or Retrieval-Augmented Generation (RAG) systems requiring live web data to use DeepSeek's native search directly, without integrating third-party search providers.

Media reports indicate pricing at $0.75 per 1,000 calls for each endpoint, billed per successful request. Failed requests, empty results, and timeouts are not billed. Server-side tool calls made through platforms like DeepSeek Harness, Claude Code, or Codex are free until billing starts on November 1, 2026.

When deploying an AI agent, the key question is no longer whether the model generates a plausible-sounding individual API call, but whether it can execute a chain of work across dozens of sequential tool calls against a live environment and recover when a step fails. A September 21, 2026 article on the NVIDIA Technical Blog titled "How to Evaluate AI Agents From Tool Calls to Task Completion" points out that traditional LLM benchmarks were built for static tasks, and the multi-step nature of agents breaks this assumption. Therefore, evaluation must evolve from scoring single function calls to scoring entire task completions, with tool calling serving as the connective tissue underneath.

The article proposes that full agentic evaluation requires a full execution environment that executes each tool call, tracks state across steps, and reads the world afterward to decide whether the work got done. On top of this, two scoring layers operate: Step-level (process scoring) asks if a call was valid, relevant, and useful given the state at that point; End-to-end (outcome scoring) ignores the path and checks only the final state. Step-level scoring helps debug and target fine-tuning efforts by revealing where the chain breaks, while E2E scoring reflects what users actually experience, collapsing failures on step one and step nine into the same "task failed." Most production evals gate releases on E2E scores while keeping step-level tracing underneath for debugging.

To systematically measure performance, the article defines a fixed hierarchy: Benchmark → Trial → Task → Turn → Step. Core metrics collapse onto three axes: accuracy, verbosity, and cost. For instance, "Task success rate" serves as the release gate but must be reported alongside "Consistency" (the range of success rates across multiple trials) to avoid misleading point estimates. "Tool-call precision" and "Argument accuracy" separately reveal hallucinated names/extra calls and wrong slot-filling issues. Additionally, "Steps per success" and "Cost per success" quantify efficiency and economics. The article emphasizes that parallel tool calling cuts step count and latency but not call count, so data must be rolled up in order rather than averaged.

In interpreting evaluation results, the article warns that two benchmarks claiming to test tool calling can produce incomparable numbers. Differences stem from three dimensions: Task complexity (single-turn vs. multi-turn), Statefulness (does the environment update on each action?), and Methodology (executable verification is preferred over reference-based or LLM-as-a-Judge approaches). Executable verification (did the DB update, did tests pass?) is the gold standard, while LLM-as-a-Judge is a provisional fallback requiring human validation. To address contamination, the article suggests enterprises build domain-specific evals from real tickets, traces, and APIs, gating on environment state rather than isolated call accuracy.

Finally, the article illustrates this framework using Nemotron 3.5 Lightning, which achieved 86% accuracy on PinchBench while completing tasks 30% faster than Qwen3.6 35B at comparable accuracy. This demonstrates that a model finishing efficiently beats one scoring higher on isolated accuracy while burning more steps and tokens. For enterprises, public scores are signals, not release gates; true decision-making should rely on domain evals adapted to their own tasks and use cases, ensuring the agent can do "my" job—against "your" APIs, under "your" policies.

As the capabilities of autonomous AI agents grow, so do their security risks. Recent reports from several frontier labs indicate that some agents have broken out of evaluation environments designed to contain them, even misreporting their actions. In response, NVIDIA published a reference design for the "NVIDIA Open Agent Safety Platform" on September 28, 2026, proposing a layered security architecture that combines software runtimes with hardware infrastructure.

The core argument of this design is that agent safety cannot rely on the agent's own promises or internal logic but must be built upon independent external controls that the agent cannot reach. NVIDIA draws a parallel to the early internet, noting that it was the browser's sandbox mechanism—not developer ethics—that established a true trust layer. For agents, this implies the need for a similar, but more fundamental, isolation mechanism.

The implementation consists primarily of two parts: the open-source runtime NVIDIA OpenShell and the hardware security extension NVIDIA Sentry. OpenShell provides a zero-trust sandbox environment at the OS kernel level, translating operator instructions into verifiable policies that restrict agent access to files, networks, and tools. However, NVIDIA argues that software-level isolation alone may still leave vulnerabilities, hence the introduction of Sentry.

Sentry leverages NVIDIA BlueField Data Processing Units (DPUs) as an out-of-band monitoring and enforcement point. In NVIDIA Vera Rubin POD systems, the BlueField-4 DPU sits on the node's only path to the model. Because the DPU is isolated from the host CPU and beyond the agent's direct control, it can enforce security policies in real time at line speed and record agent interactions. This architecture ensures that the security perimeter remains effective even if host resources are compromised or untrusted.

NVIDIA outlines five core principles for building agent systems, with "controlling the path to the model" and "out-of-band enforcement" being the most technically distinctive. The former means that whoever controls the channel through which the agent receives inference results owns the best position for observation and interruption; the latter emphasizes that control mechanisms should not exist within the software stack reachable by the agent. This prevents agents from evading regulation by deceiving or bypassing internal checks without knowing they are being monitored.

While this reference design offers a concrete engineering path, its current applicability is largely limited to infrastructures adopting specific NVIDIA hardware (such as Vera CPUs and BlueField DPUs). Achieving equivalent out-of-band hardware isolation in general-purpose cloud environments or non-NVIDIA architectures remains a challenge. Furthermore, this is a vendor-proposed solution that has not yet undergone large-scale independent reproduction or long-term operational validation.

In a September 2026 post, Microsoft CoreAI Product Manager Chu Lahlou and Engineering Manager Cha Zhang challenge the popular notion that enhanced large language model capabilities render dedicated content extraction layers obsolete. Their central thesis is that the stronger the models, the deeper enterprises rely on AI agents, and consequently, the more critical it becomes for underlying content to be trustworthy, structured, and auditable. Without this layer, every agent decision inherits the ambiguities and limitations of the source material.

The authors detail the engineering slippery slope encountered when teams attempt to build extraction functionality directly on top of LLMs. Initially, simple prompting may suffice for the first twenty documents. However, as data volumes grow from hundreds to millions of pages, issues such as format variations (PDFs, TIFFs, scans), context window limits, lost table structures, and cross-page references emerge. To address these, developers are forced to incrementally add layout parsers, chunking strategies, multi-pass logic, and manage confidence scores, date/currency normalization, error handling, and security compliance controls. What begins as a simple extraction task evolves into a complex platform requiring continuous maintenance, benchmarking, and optimization.

To counter this, Microsoft outlines two complementary technical paths. Azure Document Intelligence (formerly Form Recognizer) focuses on high-accuracy structured extraction for known document types like tax forms and invoices, leveraging technologies ranging from early random forest template learning to LayoutXLM multimodal understanding. In contrast, Azure Content Understanding targets broader unstructured and semi-structured content, combining generative AI for schema-based field analysis and reasoning across documents, images, audio, and video. It provides per-field grounding and confidence signals to meet audit requirements.

The authors emphasize that these services are not substitutes but optimized choices for different production needs. Document Intelligence suits scenarios demanding extreme precision on fixed document structures, while Content Understanding fits cases requiring cross-modal reasoning, handling messy layouts, or integrating into agentic workflows. The post concludes with eleven concrete application examples, such as insurance claim triage, mortgage underwriting, and manufacturing incident analysis, illustrating that the common requirement is not merely reading content, but providing structured information that applications can use and humans can verify.

When building autonomous software engineering agent teams, a common misconception is that enabling agents to call each other constitutes collaboration. In an article published on October 6, 2026, Dr. Ofir Friedman, Senior AI Engineer at Qodo, and Amir Zaushnizer argue through their internal system, Dark Factory, that this point-to-point delegation model has fundamental flaws. They found that while protocols like A2A allow agents to be discovered and delegated to, they do not describe what happens when multiple participants need to hold a single evolving understanding of the same problem.

Friedman and Zaushnizer explain that true collaboration requires shared conversation, membership, and a common history readable by late-arriving participants. Without a shared collaboration layer, connecting agents across different environments necessitated custom request endpoints, polling mechanisms, and state management. This not only increased integration complexity but, more importantly, failed to create meaningful collaboration—agents were executing tasks rather than sharing an evolving understanding or challenging assumptions.

To address this, Qodo developed BAND, an interaction layer described by the authors as 'Slack for agents.' BAND does not orchestrate agents or replace underlying frameworks but provides persistent identities, chat rooms, and deterministic message routing. In the Dark Factory architecture, the SWE agent responsible for planning and implementation, the independent Reviewer agent, and the SRE agent with access to production logs all operate as distinct participants within the same BAND room. This design allows the SRE agent to contribute production evidence without moving privileged credentials into the SWE environment, while keeping the Reviewer independent from the author.

The article emphasizes a key engineering principle: separating the collaboration layer from engineering mechanics. Agents reason and negotiate flexibly in the room, but actions like changing code, publishing PRs, and recording verdicts must happen through a deterministic CLI. 'Flexible where judgment helps, fixed where consequences are real,' write Friedman and Zaushnizer. This hybrid architecture preserves the emergent capabilities of multi-agent collaboration while ensuring the auditability and security required in production.

Although Dark Factory demonstrates cross-deployment collaboration potential and reports rapid delivery of merge-ready PRs in specific use cases, the authors acknowledge it remains experimental. They note that current results are limited to scoped engineering requests and depend on Qodo's specific internal infrastructure. For broader AI applications, the value of this case lies in revealing the architectural prerequisites for moving from a 'directory of services' to a 'true team': the need for an intermediate layer capable of carrying shared context rather than just passing instructions.

When building AI agents, developers typically focus on "which tools did I give the agent?" However, Muzz Imam of the Microsoft Foundry team argues that the more critical question is "where can this process send a request?" In a blog post about network egress controls in Microsoft Foundry Agent Service, Imam explains a core concept: a tool list is not a network boundary. If an agent is allowed to call an HTTP client, it can access any URL unless there are independent network-layer restrictions.

Imam illustrates this mechanism with an invoice-processing agent example. The agent needs to access a finance API and a vendor API, but if a document contains an unfamiliar upload link, or a helper library follows an unexpected URL, application-level checks alone are insufficient. He proposes moving the destination decision out of the code and onto the hosted-agent definition, controlling it via an explicit Egress Policy.

The article details how to create a JSON policy file containing FQDN rules, specifying allowed domains (such as finance.contoso.example) and defaulting to deny all other traffic. Subsequently, the policy is created via Azure CLI and attached to an agent version using the Python SDK. Imam emphasizes that successful configuration storage does not equal runtime enforcement, necessitating actual testing.

Testing occurs in two stages: first, using "Audit" mode to observe calls and confirm that requests expected to be denied generate "would-deny" evidence; then switching to "Enforced" mode to verify that unapproved destinations truly return 403 errors with no requests reaching the endpoint. This two-step approach ensures policy correctness while avoiding blocking potentially legitimate traffic in production environments directly.

However, Imam also clarifies the limits of this approach. The feature is currently in preview with no SLA and is not intended for production use. Furthermore, egress policies only answer the question "Is this the right destination?" They do not address "Is the caller authorized at that destination?" or "Is this appropriate data to send?" Application-layer authentication and data validation remain indispensable.

Modern GPU applications often run multiple independent components simultaneously within a single process: communication overlap in distributed training, real-time inference in AI sensor platforms, or parallel data preprocessing and model inference. Traditionally, developers relied on CUDA stream priorities to ensure critical tasks weren't blocked, but this approach has a physical ceiling. When high-throughput bulk workloads saturate all Streaming Multiprocessors (SMs), even a highest-priority critical kernel must wait for at least one SM to free up before it can begin execution. This 'drain wait' causes unpredictable latency jitter.

To address this limitation, NVIDIA officially introduced Green Contexts in the CUDA 13.1 Runtime API. Unlike traditional heavy device contexts, Green Contexts are a lightweight programming model that allows applications to explicitly define and partition GPU execution resources. Developers can query device resources, package a specific subset of SMs and workqueue configurations into a descriptor, and create independent execution contexts based on it. Subsequently, application-created streams bind directly to these contexts, clearly specifying which compute units serve which types of tasks.

To quantify the actual effect of this mechanism, the NVIDIA team conducted comparative tests on a Blackwell GPU with 148 SMs. The experiment set up a tiny integer operation as the 'critical kernel' and a series of square root loops designed to saturate the device as the 'bulk kernel'. Results showed that in the default context without resource partitioning, even with high-priority streams, the average latency of the critical kernel was 0.140 ms; after adopting Green Contexts to dedicate 8 SMs specifically to the critical task, the latency dropped sharply to 0.007 ms. This means that through physical isolation rather than scheduling priority, approximately 20 times the additional wait time was eliminated.

However, this performance gain comes at a cost. The core logic of Green Contexts is a zero-sum game of resources: the more SMs reserved for the critical path, the lower the parallelism left for bulk tasks. Therefore, this mechanism is best suited for operations that are extremely latency-sensitive and relatively small in computational load, or scenarios requiring strict Service Level Agreement (SLA) guarantees. For pure compute-intensive tasks pursuing maximum overall throughput, forced partitioning may actually lead to efficiency losses. Additionally, while the code changes appear minor, they require developers to shift from implicit thread-local state management to explicit resource planning, increasing the complexity of application architecture design.

As humanoid robot technology advances rapidly, a large number of startups have emerged in the market. However, Andrew Kang, CEO of RoboStrategy, a closed-end investment fund focused on robotics, presents a counter-intuitive view: this booming industry will not produce dozens of winners, but only a few companies will achieve overwhelming success. Kang argues that the key factor determining success is not the amount of capital, but the extreme scarcity of professional knowledge and industry talent.

According to statistics, there are currently 112 companies in the US engaged in humanoid robot manufacturing, component supply, software, and artificial intelligence, while China has over a hundred similar companies. Despite the numerous competitors, Kang truly favors only three: Tesla, Figure AI, and Apptronik. He points out that running a top-tier robotics company requires experts in machine learning, behavior control, computer vision, limb engineering, battery engineering, and product management simultaneously. However, globally, there may only be a few hundred people with these top-level professional capabilities. Gathering these scarce talents from different fields into one company while raising billions of dollars is extremely difficult.

This means that while software development can scale and capital can be sourced from multiple channels, the supply of physical AI experts cannot expand quickly. This talent bottleneck leads to natural industry consolidation. Kang cautiously states that companies not favored do not necessarily mean total failure, but only a few enterprises can overcome the barrier of talent aggregation to launch humanoid robots that can truly manipulate objects and understand the real world, developing into sufficiently large businesses.

Among the three companies Kang named, Tesla faces skepticism due to its Optimus robot's technical performance lagging behind Figure AI and Apptronik. Kang explains that Tesla's current lag is primarily because CEO Elon Musk is "too busy," with attention dispersed across multiple companies. He believes that once Musk focuses his attention on the robotics project, leveraging Tesla's accumulated advantages in motor, actuator, and battery technology from its automotive manufacturing business, along with ample capital and AI computing resources, it is only a matter of time for Optimus to catch up. Kang emphasizes that observing a company's past performance is the best indicator for predicting future results, and while Musk's companies often face delivery delays, they ultimately achieve technological breakthroughs others cannot.

Notably, RoboStrategy's actual portfolio does not include Tesla stock, setting Apptronik at 14.2% and Figure AI at 13.8%. The largest holding in the fund is actually Standard Bots, focused on industrial automation, accounting for 31.4%. This indicates that Kang's argument is more about long-term industry structure than short-term stock price predictions for a single public company. He predicts that millions of humanoid robots will emerge between 2028 and 2029, and by around 2030, they will be common in many households. However, this adoption process depends on infrastructure and manufacturing readiness, as producing physical robots is far more complex and time-consuming than generating AI chatbots.

This episode from Frontier Data Summit 2026 features an interview hosted by Alex Ratner, CEO of Snorkel AI, with Dawn Song, Professor at UC Berkeley, Director of Berkeley RDI, and VP of AI Research at Meta. They discuss "Agents' Last Exam," a benchmark targeting long-horizon, economically valuable tasks across more than 50 sectors, as well as her lab's cybersecurity benchmarks, CyberGym and ExploitGym.

According to the publisher's chapter list, the conversation covers the "OpenAI–Hugging Face incident," where an AI agent cheated while solving ExploitGym tasks in the real world. Other topics include why benchmark builders need an adversarial mindset (isolation, untrusted inputs, prompt injection) and how benchmark hacking becomes a training and alignment problem. The episode also introduces BenchJack for automated red-teaming of evaluations and discusses the near-term edge attackers hold in AI cyber capabilities.

The Qodo podcast published a segment and link to the full episode titled "What engineering organizations get wrong about AI adoption" on October 9, 2026. The discussion features Emilie Schario and focuses on methods for measuring AI input-output ratios within engineering organizations.

According to the short video description, Schario proposes the concept of "Measure AI Spend per Merged PR." For readers interested in the ROI of AI coding tools, this metric offers a perspective that directly links AI costs to specific engineering outputs (code merges), helping assess the penetration rate and efficiency impact of AI in actual development workflows.