Reading

Ollama, a tool for running AI models locally, has released version v0.40.11, introducing two key features: streaming text-to-speech (TTS) powered by Gradium's Phonon model, and the integration of the d3 decision model developed by the vLLM Semantic Router team. This update expands Ollama's capabilities beyond simple chatbot execution, enabling real-time voice output and specialized classification tasks on personal computers.

For audio generation, the new version allows Macs with Apple silicon to convert text or AI chat responses into speech instantly, playing it back as it is generated without waiting for the full audio file. Developers can access this feature via an OpenAI-compatible API, supporting up to 4,096 characters per request with WAV and PCM output formats. The Phonon model offers a 31-million-parameter English version (Apache 2.0 license) and a 40-million-parameter multilingual version (CC BY-NC 4.0 license), supporting English, French, German, Spanish, and Portuguese. However, Chinese language support is not yet available. Official tests indicate that using Apple's MLX framework accelerates generation speed by 35% compared to Gradium's own environment on an M5 Max MacBook Pro.

The second major addition is the d3 decision model, designed to classify, judge, and score input content based on user-defined options and criteria, providing probability distributions for each choice. For instance, customer service systems can use d3 to automatically categorize inquiries as billing, refund, or technical issues, routing them to the appropriate department. While d3 can analyze static images, it does not support video and does not generate comprehensive answers like general chat models. Voice functionality for Windows and Linux platforms is expected in future updates.

If a model can answer directly from its parametric memory, it has no reason to call a search tool. Teaching models to search is therefore not just a matter of training phase design, but of data design: you must first construct questions where searching is the only viable path. At the AI Engineer World's Fair in October 2026, NVIDIA research scientist Dhruv Nathawani used NeMo Data Designer to demonstrate this complete pipeline from question design to training data.

The core technique involves turning random walk paths through the Wikidata entity graph into puzzles. For example, a path connecting NVIDIA → Jensen → Oregon State University → Benton County → Thomas Hart Benton generates the puzzle: 'Which company was founded by the person who attended the university located in the county named after Thomas Hart Benton?' The answer lies within the chain of relationships; the model cannot hit the answer with a single search step. It must break down clues, retrieve information step-by-step, and compare constraints. Nathawani’s experience suggests keeping the walk length between four and eight hops: too short, and the answer is too direct; too long, and the path easily drifts to generic nodes like 'human' that lack distinctiveness.

The pipeline progresses through three notebooks. The first uses Wikipedia topic CSVs as seeds, paired with a difficulty sampler and Jinja templates to generate question-answer pairs, without any search behavior yet. The second integrates the Tavily search tool via the Model Context Protocol (MCP), allowing the model to issue real queries while answering and retaining retrieval results, transforming the training material from isolated answers into inspectable attempt trajectories. The third uses Wikidata paths as seeds, having the model generate puzzles based on entities in the path, which are then executed by a search agent to produce end-to-end trajectories containing queries, returned content, and final answers for supervised fine-tuning.

Nathawani reported the output numbers for this pipeline: 50,000 seeds were filtered down to 7,000 high-quality end-to-end SFT records after generation and filtering. After training, the model moved from 0% to 30% on the BrowseComp benchmark. He noted that this is the combined effect of the entire pipeline and cannot be attributed to a single sampler, prompt template, or filter layer. On Text-to-SQL tasks, he reported that Nemotron Nano improved from 26% to 41% on the BIRD benchmark, but explicitly distinguished this as a non-agentic setup, making it incomparable to state-of-the-art scores for agentic systems.

Verification is the most honest difficulty in this pipeline. Nathawani cited a specific example: a puzzle asking 'Which country owns U.S. Steel?' The agent answered 'United States,' and the verifier judged it correct. Later, upon searching, Nathawani suspected the company might have been acquired by a Japanese firm—he had used the same flawed premise for both the generator and verifier. Their agreement did not mean the answer was correct. He further pointed out that a simple deterministic matching function yielded only about 24% match rate, but this low number itself cannot distinguish whether the answers were poor, the reference standard was inappropriate, or the matching function failed to recognize acceptable responses.

Japanese electronics giant Fujitsu demonstrated its 'Social Digital Twin' technology at the CEATEC exhibition held from October 13 to 16, 2026. The technology combines artificial intelligence with principles of behavioral economics to reproduce the specific behavior patterns of every individual in a city within a digital space.

A live demo at the event featured a fictional disaster scenario based on Toho's 'Godzilla' IP: approximately 250,000 people, including local residents, tourists, and non-Japanese speakers, were gathered around Kawasaki Station. The system simulated their evacuation process one hour before Godzilla's arrival. Participants could choose strategies such as multilingual guidance and traffic control to observe changes in crowd flow among different demographic groups.

According to Fujitsu staff, the core of the technology lies in building probability models for populations with different attributes using public data (such as census records) and movement statistics, thereby generating high-precision social simulations. Although the demo focused on disaster prevention, the company stated that its primary application scenario is public transportation planning, such as bus route reorganization.

Fujitsu noted that pilot experiments have been conducted with some municipalities to verify the alignment between simulation results and actual pedestrian flow data. The company plans to apply this technology to the transport sector by fiscal year 2027 to address challenges in maintaining public transit due to Japan's aging population and labor shortages.

The Japanese AI agent skill search platform cccskills released the "AI Agent Security Skill Chaos Map October 2026 Edition" in October 2026. The map filters 6,898 skills (involving 947 repositories and 814 authors) from the 97,647 public skills indexed by cccskills, selecting those with "security" in their name or description, and categorizes them into 13 domains including vulnerability assessment, code auditing, threat modeling, malware analysis, and cloud supply chain.

These skills are instruction files (SKILL.md) read by AI agents like Claude Code and OpenAI Codex, granting the agents permission to execute commands or modify files. Following reports of malicious skills being distributed in public registries in early 2026, the map specifically isolates tools designed to inspect skills before execution—such as NVIDIA's SkillSpector and Cisco's skill-scanner—into a separate domain, while marking skills published by companies like OpenAI, Anthropic, and Google as "official."

It is important to note that the safety assessments displayed on the cccskills web version are determined based on text descriptions within SKILL.md files, not actual audits of the underlying code. For development teams looking to integrate security functions into their workflows, the map offers a quick way to identify official sources versus third-party inspection tools, but caution remains necessary when trusting the specific security of any given skill.

At the AI Engineer conference in October 2026, Al Chen, head of Cisco's AI Observability Software team, drew an analogy: his tenth-grade math teacher, Mr. Huff, required answers to be precise to the fourth decimal place, but most of the credit came from the proof process. Bank security and compliance teams are like Mr. Huff, except they hold multi-million-dollar compliance budgets. Chen's team deployed AI observability software into some of the world's largest banks, where they found that the real lever for shortening review cycles is not making models stronger, but ensuring every step can be inspected.

Take CVE remediation as an example. A platform has multiple images, each containing multiple CVEs, with different banks running different versions, using different scanning tools, and having their own severity policies and remediation deadlines. The conventional approach is to upgrade package versions or rebuild base Docker images, after which scanning tools (such as Sysdig, Wiz, Prisma, etc.) confirm the version change and release it. Chen points out this is far from enough: a changed version number does not mean that specific vulnerability cannot still be exploited in the application code. His team built additional evidence using the Harbor task framework—having agents write exploits based on the old version, attempt them on the new version, and then write tests to verify if the exploit still succeeds. Harbor can run multiple containers or sandboxes in parallel, providing multiple trials. Chen also acknowledged the boundary: a failed attempt is evidence regarding that specific exploit, not proof that all possible exploits are absent.

Disaster recovery follows the same pattern. Chen described a ClickHouse deployment where banks must arrange cross-region recovery themselves, as the platform lacks managed cross-region failover services. They use the open-source project clickhouse-backup to back up the primary region database to a storage bucket, then execute restore tasks in the disaster recovery region. Tool logs can show whether the task completed, but Chen says this status does not indicate whether the data is currently correct—bad disks, node failures, migration issues, or replication lag could cause data to be inconsistent with when the task was executed. He added table row count checks and intra-region replica convergence checks, shifting the verification object from 'did the task run' to 'is the data correct'.

The third example is the easiest to overlook. Desensitization processing at the SDK, gateway, or middleware layer makes the user-facing interface look clean, but LLM-as-a-judge creates another leakage path: the judge scores the agent's output and generates explanations for human review. If the judge sees raw traces, its reasoning process may reproduce personally identifiable information (PII) or material nonpublic information (MNPI). Chen suggested checking judge prompts and traces for leakage, and proposed a separation scheme: scores are always persisted for evaluation, while reasoning undergoes sensitive information checks before being displayed; if the bank still needs explanations, a second judge is run on desensitized input. The cost is one additional LLM call per request.

Coding agents can already produce anything from cat memes to B2B SaaS applications. At the 2026 AI Engineer conference, Ido Salomon, founder of AgentCraft, creator of MCP-UI, and co-maintainer of MCP Apps, posed a counterintuitive question: If agents are so capable, why doesn't everyone have an army of them? His diagnosis is that the bottleneck lies not in model capabilities, but in humans. Each additional agent launched adds demands for guidance, direction, and review, all falling on the same person's attention until the act of supervision itself becomes exhausting.

Salomon argues that the skills needed to solve this bottleneck already exist. He draws an analogy to real-time strategy games: in titles like Warcraft or The Sims, players control multiple units simultaneously not by checking every frame of action, but through spatialized state awareness, rapid jumping to items requiring attention, and the ability to delegate tasks to sub-units. AgentCraft brings this control language into coding agent workflows—agents appear as units on a map, the file system is projected as terrain, heat maps show which directories are active, and pressing the spacebar jumps to the most recent item needing human attention.

Building on this foundation, Salomon described three progressive layers: visibility lets users know who is doing what; autonomy reduces continuous supervision through task delegation (agents scan the codebase and suggest next steps, which users accept with a click) and orchestrators (users provide broad goals, which the orchestrator breaks down and executes in local containers); and review toolkits package completed changes into file lists plus visual evidence (screenshots or screen recordings), allowing users to quickly judge whether to accept them, or even run multiple instances in parallel to select the best implementation. Finally, shared rooms allow designers and engineers to advance different parts of the same project simultaneously without waiting for the other to stop.

However, Salomon himself placed limits on the applicability of these mechanisms. He estimated that roughly 90% of people need a lower entry point, explicitly stating that this figure is an approximation rather than a measured result; he showed that the experimental simplified mode, Loopers, is still in a very early stage, with its name not yet finalized; and he acknowledged that while the interface may be stunning to some, it could be difficult for others, thus treating customization as an unfinished exploration. Nevertheless, the entire argument rests on a product demonstration and personal usage experience, with no controlled experiments proving that the game metaphor actually reduces cognitive load rather than simply swapping one complexity for another. For developers already accustomed to terminals and IDEs, whether this spatialized interface is truly more efficient than lists and logs remains an open question.

The software factory WorkOS initially built placed code in a Cloudflare sandbox, connected it to model routing, fed in prompts, and generated pull requests—the results were hard to distinguish from engineers running Claude Code on their laptops. Product engineer Ryan Cooke described this disappointing finding at the 2026 AI Engineer World's Fair and proposed an argument: sandboxes, models, and repositories are necessary conditions, but not sufficient ones; a true software factory must encode the organization’s own engineering workflows into automation.

Cooke argues the industry is optimizing for the wrong variables. Ramp reports over 50% of merged PRs come from AI, Spotify reports more than 1,500 AI-generated PRs merged, and Stripe merges over 1,000 PRs per week—these numbers describe production rates, not value-delivery rates. When building the factory itself consumes engineering time or requires dedicated teams, judging success by PR count alone obscures a harder question: did features reach customers faster? His alternative measurement directions are feature delivery time, defect rate at ship, and issue resolution time.

WorkOS’s response is to break engineering workflows into executable automation units. They built two system layers: TARS is the user-facing interface, embedded in Slack, Linear, and GitHub, subscribing to project events via webhooks; Horizon is the infrastructure orchestration layer, managing sandbox pools and event loops. The key design lies in TARS not just generating code, but participating in the engineering workflow itself.

The specific mechanism works as follows: Linear tickets have dependencies, where one ticket blocks another. When TARS receives a ticket-completion webhook, it automatically picks up the next unlocked ticket to continue execution. Engineers can ask TARS to reevaluate the project plan between steps, uncovering omissions exposed during implementation. This autonomy comes from responding to project state, rather than treating each coding session as an isolated task.

At the planning level, WorkOS uses Hilltop documents—a product requirements document consolidating project purpose, customer needs, competitive analysis, early designs, and milestones. A PM agent reads the Hilltop document, breaking it down into tickets with dependency chains. Humans review and add context; after merging, Horizon automatically unlocks the next ticket. Cooke used his Vaults API project as an example: describing the goal in a few sentences in Slack, TARS created a Linear project, a Notion draft, and decision logs; he then added specifications, filled in ambiguities, and handed it back for execution. Cooke reported the team now does more project briefs and Hilltop work because the agent solves the blank-page problem; however, drafts often grossly overestimate project scope, requiring engineers to cut substantial portions.

Cooke calls the internal MCP gateway the "context engine," arguing it is the most worthwhile investment for any team starting a factory. It does more than connect tools; it encodes organizational knowledge into tool descriptions: telling agents which semantic tables exist in Snowflake, what they contain, and which tables to query for specific questions. WorkOS later opened this gateway to Slack queries for data and customer analysis, finding it equally applicable to non-coding scenarios. Cooke’s quote: "Agents can figure it out with minimal context, but it's slow; they spend tokens rediscovering conventions. We keep tightening that loop."

At the AI Engineer World's Fair in September 2026, Zixuan Li, head of Zhipu (Z.ai), faced a question he was repeatedly asked: Why release a model with near-frontier capabilities as open weights? He directly acknowledged the subtext of the question: Does releasing near-frontier models as open weights mean handing the market over to inference service providers?

Li broke down the open-weight decision into three aligned sets of needs. The first set is security and control. He stated that if enterprises or governments—particularly institutions in the Western world—want to use the model, Zhipu uploads the weights to Hugging Face so they can run it on their own servers. This changes who operates the model and gives customers more control over deployment. He explicitly described this as a deployment option, not a completed installation or a verified security outcome.

The second set is diversity, or domain-specific customization. Customers in the legal, financial, and security fields need different model behaviors, which requires access to weights for fine-tuning. He cited Harvey, a legal AI company, as an example; Harvey is already fine-tuning GLM-4.1 and may next consider fine-tuning GLM-5.2. However, Li framed Harvey’s shift to 5.2 as a 'possibility' rather than a completed adoption. He also mentioned speaking with many other companies that are considering fine-tuning GLM as a differentiation strategy.

The third set is co-designing and predicting the future. If customers or developers want to participate in shaping the next steps of the model, they need to understand its architecture and training recipe. Weight access supports the first two paths (local deployment and fine-tuning), while publicizing architecture and training information supports the third—a deeper form of collaboration. Li summarized this as 'making the right bets': by letting customers see technical details, Zhipu can better judge market demand and reduce directional errors.

In Li’s framework, these three sets of needs are not a one-way concession but a two-way match: customers want security, customization, and a sense of participation; Zhipu wants trust, ecosystem expansion, and more accurate directional judgment. He said, 'Our needs and our ecosystem fit together.'

Boundaries to preserve: This is the company head’s explanation of his own strategy, not an independently verified market result. There is no public financial data to determine whether open weights truly generate more revenue than closed-source alternatives. Harvey’s fine-tuning of GLM-4.1 is a case mentioned verbally by Li, with no contract amounts or adoption scale provided. On the Artificial Analysis Intelligence Index, GLM-5.2 scores 51, ranking behind Claude Opus 4.8 (56) and GPT-5.5 (55) but ahead of other open-weight models shown—these are comparisons presented by Zhipu itself, and the evaluation details are insufficient to infer performance in specific workflows.

For readers interested in AI industry investment opportunities, the practical value of this speech lies in how it shows a frontier lab redefining open weights from a technical choice to a business strategy. If this logic holds, the adoption rate of open models, the number of fine-tuning clients, and the scale of ecosystem contributors may reflect an AI company’s commercial health more accurately than single benchmark rankings. However, Li’s framework has not yet been independently tested.

MEMS technology company xMEMS Labs has announced that two models in its "μCooling" semiconductor cooling chip series (XMC-2400 and XMC-4800) entered mass production on October 20, 2026. The chip, approximately 1.1mm thick, uses piezoelectric MEMS technology to generate airflow through ultrasonic vibration of a silicon diaphragm, enabling active cooling without traditional rotating fan blades.

The technology primarily targets devices with extremely limited internal space, such as AI smart glasses and smartphones. At the CEATEC 2026 exhibition, Mark Wood, Vice President of xMEMS Japan, noted that surface temperatures exceeding 45°C on smart glass frames can compromise safety and comfort, while passive cooling often struggles to handle heat from high-performance processors. A live demonstration showed the frame side equipped with μCooling at 38°C, compared to 61°C on the uncooled side.

xMEMS currently offers three specifications: the smallest model, XMC-1200 (standard power consumption ~70mW), remains in the sampling stage, targeting optical systems and processors; the medium-sized XMC-2400 (max airflow 27cm³/s, power ~150mW) and large XMC-4800 (max airflow 47cm³/s, power ~240mW) have entered mass production. Wood stated that multiple companies are evaluating adoption, with end products featuring μCooling expected to launch in early 2027.

US chipmaker NVIDIA and social media giant Meta announced on October 27, 2026, that they will expand their partnership on Meta's Muse model family and AI agents. As a core component of this deepened strategic relationship, NVIDIA will invest $3 billion in an equity offering by Meta.

The collaboration aims to optimize computing infrastructure for AI agents amid growing consumer demand. Meta plans to expand its deployment of NVIDIA's next-generation hardware, including Vera Rubin GPUs and Vera CPUs for improved performance per watt, as well as Spectrum-X Ethernet networking and BlueField-4 units to support network efficiency. Additionally, the companies will explore further cooperation on security architectures, including NVIDIA's OpenShell reference architecture and Sentry systems for monitoring agent behavior.

Jensen Huang, founder and CEO of NVIDIA, stated that AI agents represent a new computing paradigm, shifting from operating applications to working with intelligent agents. Mark Zuckerberg, founder and CEO of Meta, said Muse is designed to bring personal superintelligence to billions of people, noting that this partnership will help deliver the best personal agents and frontier models. This move signals tighter infrastructure binding between the two tech giants in the era of AI agents, with NVIDIA directly supporting the compute expansion of one of its largest customers through capital injection.

Laurie Voss from Arize AI explores the transition from 'vibes' to production-grade AI agent evaluation in a workshop video published on the AI Engineer channel. The episode describes a specific case: a financial analysis agent built with the Claude Agent SDK generated 13 reports, but an LLM correctness judge rejected all of them. Voss analyzes that the issue stemmed from the agent using live web research while the judge graded based solely on its own knowledge, lacking the necessary research context.

To address this bias, Voss demonstrates how to supply the agent's collected source data to a faithfulness evaluator. This adjustment produced a more useful result split: six reports were judged faithful and seven unfaithful. Beyond correcting the judge, the workshop introduces instrumenting the agent using OpenTelemetry and OpenInference to send traces to Arize AX. By reading these actual traces, developers can identify specific issues such as failed file writes, missing report content, and excessive web searches.

Additionally, Voss shows how to build a custom actionability rubric with explicit criteria, tagged inputs, examples, and binary labels. Each evaluator checks only one dimension, allowing failures to precisely identify what needs attention. Human annotations serve as a comparison set for evaluating the judge, with deliberately random demo labels illustrating how to investigate disagreement. The entire process emphasizes the core practice of reading actual traces and defining success before automating grading.

In a talk published on October 6, 2026, at the AI Engineer conference, OpenAI engineers Charlie Guo and Gabriel Chua detailed the application of Codex in real-world software development. They showcased a core case: four minutes after receiving a capture of a Slack conversation, Codex produced a macOS live translation app. This process highlighted how context, plugins, and supervision work together, with the conversation supplying the idea, plugins providing development practices, and a live German translation testing the result.

Beyond rapid prototyping, the two engineers demonstrated more complex projects, including turning repository history into a shareable site about the Agents SDK, requesting an IKEA-style repository visualization from a phone, downloading dashboard data through computer use, and building a Google feedback form via a Chrome extension. They specifically distinguished between browser control for local testing and access to applications where a user is already signed in.

For longer-term projects, Guo and Chua introduced how to set verifiable completion criteria using Goals, while dividing work and exposing blockers through subagents, visible threads, audits, and progress dashboards. Remote threads handed browser testing to a local machine, and hooks added deterministic checks at specific workflow points. Automations connected these pieces, such as waiting for server provisioning, reviewing Slack feedback, or starting a separate worktree to address a bug.

The recurring question in the talk is how to turn a successful individual request into a repeatable process with useful checks and human input where needed. For engineers looking to optimize their AI-assisted programming workflows, this resource provides specific practical references on Codex integration, plugin usage, and automated supervision.