LecturaInvestigaciónRadarMarco de inversión
Iniciar sesión / Registrarse
TodoTecnologíaProductosInversiónBolsaOtros

Lectura

2026-11-2154 publicaciones

Actualizar

Gemini Enterprise: What Authentication a Custom UI Inherits From the Platform

Resumen rápido
2026-11-20

["When enterprises build a custom interface on top of Gemini Enterprise rather than using its built-in web app, they inherit three authentication flows the platform previously handled automatically. Neeraj Agrawal, a Forward Deployed Engineer at Google Cloud, walks through each layer in a November 2026 guide: authenticating the user to the StreamAssist API (via Google Identity, Workforce Identity Federation, or Workload Identity Federation), authorizing Federated Connectors through 3-Legged OAuth with tokens stored in Gemini Enterprise's managed Keystore, and handling per-tool OAuth for custom ADK agents deployed on Vertex AI Agent Engine.","The most transferable insight concerns the third layer. Agrawal explains that attaching static OAuth configurations at the root agent registration level blocks the entire agent upfront — a user cannot ask a general policy question without being forced through every connector login. The solution is wrapping protected sub-agent tools with GcpAuthProvider backed by GCP Auth Manager's Token Vault, deploying with Agent Identity enabled, and triggering 3LO popups only when a protected tool is actually called. After the first authorization, subsequent queries run in a single turn with no popups.","This is vendor-authored guidance for Google's own product. The specific components (GcpAuthProvider, AuthenticatedFunctionTool, SPIFFE identity, Keystore) are tightly coupled to Google Cloud infrastructure, and no independent validation of the pattern's effectiveness or adoption beyond this ecosystem is provided."]

Fuentes:developers.googleblog.com

Investigación

Private MCP Connectivity for Gemini Enterprise: Egress Gateway, PSC Network Attachment, and FQDN-Scoped DNS

Material
2026-11-20

When an enterprise moves a custom MCP server from prototype to production on Cloud Run, three constraints collide simultaneously. Security policy demands internal-only ingress, blocking all public traffic. The platform requires both service-level and user-level authentication on every tool call. And in large Shared VPCs hosting hundreds of Cloud Run services, a naive DNS override can hijack resolution for every other service on the network. Neeraj Agrawal, a Forward Deployed Engineer at Google Cloud, describes a three-part architecture that resolves all three at once.

The first component is an Egress Agent Gateway in Agent-to-Anywhere mode, connected via a PSC Network Attachment to a regional subnet. This places a managed network interface inside the enterprise VPC without public IPs or inbound firewall rules — the packet enters the private network at the attachment point rather than traversing the public internet. From inside the VPC, the request reaches Cloud Run's serverless frontend through either Private Google Access virtual IPs (199.36.153.8/30) or a global PSC endpoint (240.0.0.10), depending on whether the subnet retains a default internet route.

The second component addresses the DNS problem specifically. Without Private Cloud DNS, public resolution maps .run.app domains to public IPs that time out because the PSC Network Attachment does not route to the internet. The intuitive fix — a wildcard *.run.app private zone — breaks every other Cloud Run service in the Shared VPC. The solution Agrawal describes is to create a Private Cloud DNS zone scoped exclusively to the single MCP service's exact fully qualified domain name, then peer only that domain with the Agent Gateway. Because Cloud DNS uses longest-suffix matching, all other services continue resolving through their normal paths.

The third component handles authentication. Gemini Enterprise sends two distinct headers on every request: an X-Serverless-Authorization header containing a Google-signed OIDC ID token for the service agent, which Cloud Run IAM verifies at the serverless edge and strips before the request reaches the container; and an Authorization Bearer header containing the signed-in user's OAuth 2.0 access token, which passes untouched to the MCP application code for per-user permission enforcement. The provided Python middleware example shows how to validate the user token while allowing health checks to bypass authentication.

The architecture is documented in a companion GitHub repository with sequential setup scripts covering both the Private Google Access and global PSC endpoint options, along with Cloud Console instructions and Cloud Logging verification steps.

Fuentes:developers.googleblog.com

Investigación

How vLLM Serves a Looped Transformer: Per-Pass Caches as Regular Layers, Fused Message Reads, and a Variable-Pass Tradeoff Table

Material
2026-11-20

Inkling 2 Mini's architecture creates a concrete serving problem: its 24 physical layers split into a prelude (layers 0–5), a loop core (layers 6–17) and a coda (layers 18–23), and the core runs three times per token with shared weights. Each pass maintains its own KV cache and reads gated messages from the pass before it. The model therefore has 48 KV caches and 48 conv-state caches despite having only 24 layers.

vLLM's solution is to register each extra pass of a core layer as its own attention layer with its own KV and conv-state cache. The KV cache manager and prefix caching then treat these caches like any others, so serving the loop needed no new cache-management logic. This is the central engineering decision: the loop is absorbed into the existing layer abstraction rather than requiring a new execution path.

The cross-pass messages are the second challenge. From the second pass onward, each core sublayer input — the attention's query, key and value inputs and the MLP input — gets its own message, read from the previous pass's two blocks by an attention-residual read with a learned query per layer and input type. A rank-64 controller then merges the message into the input using a gated formula: the output is (1 − tanh(W_forget · h)) · input + tanh(W_gain · h) · message, where h is computed from RMS-normalized input and message via a down-projection. With 12 core layers and 4 inputs each, every later pass reads 48 messages per token.

A direct implementation would launch 4 message reads and 4 controllers, several kernels each, for every core layer in a later pass. vLLM instead reads all four messages of a layer in one attention-residual kernel and stacks the four controllers' weights, so their down and gain/forget projections run as batched GEMMs. One Triton kernel computes the controllers' RMS normalizations, and another fuses the gated merge with the RMSNorm of the sublayer the message feeds. The merge runs in FP32 without FMA contraction, matching the reference implementation's numerics.

For the layers themselves, vLLM reuses Inkling 2 Small's fused kernels — each sublayer's all-reduce, short convolution, residual update, attention-residual mix and RMSNorm fused into one kernel — generalized from hidden size 4,608 to 2,304, with a new mode that returns the FP32 mix before the RMSNorm so later passes can merge their messages into it.

Thinking Machines also released TML-d24-varloop-research, a checkpoint trained to run with a variable number of passes. vLLM exposes two options through --hf-overrides: loop_num_passes (at least 1, default 3) and loop_share_cache (all passes of a core layer share one KV cache, each overwriting the previous pass's keys and values; conv states stay per pass). On one GB200 at TP1 with 4K context and forward pass only, the measured decode-step times are 1.51 ms for 1 pass, 2.67 ms for 2 passes, and 3.85 ms for 3 passes at batch size 1; at batch 32, 3.81 ms, 5.92 ms and 8.11 ms respectively. KV cache capacity drops from 352K tokens at 1 pass to 175K at 3 passes. With 3 passes and a shared KV cache, decode time is essentially unchanged (3.86 ms at batch 1) but capacity recovers to 194K tokens — sharing frees less memory than dropping a pass because the per-pass conv-state caches remain.

Fuentes:vllm.ai

Investigación

Thinking Machines Lab Releases Inkling 2 Mini: A Looped MoE Model for Local AI

Material
2026-11-20 08:00 GMT+8

On November 20, 2026, Thinking Machines Lab released Inkling 2 Mini, a new multimodal Mixture-of-Experts (MoE) model designed specifically for local AI setups. The model features approximately 26 billion total parameters and 3 billion active parameters per token, supporting a context window of up to 1 million tokens and accepting both text and image inputs.

The key architectural innovation is a "looped" transformer design. While the model has 24 physical decoder layers, the middle 12 layers are visited three times in a loop during inference. This allows the model to execute 48 logical layer visits per token, effectively increasing its capacity without increasing the number of stored weights. Information moves between these passes through cross-pass messages, where each pass refines the computation of the previous one using learned gating controllers.

Inkling 2 Mini uses mixed-precision formats (NVFP4 and MXFP8 expert weights) and includes a DSpark drafter for speed-up. It supports day-zero integration with major frameworks including Transformers, Llama App (llama.cpp), MLX, and SGLang. A research variant with variable loop passes is also available, allowing users to trade off inference speed against generation quality by adjusting the number of loops.

According to vendor self-reported benchmarks, Inkling 2 Mini leads in 32 out of 48 tests compared to models like Gemma 4 31B, Qwen3.8-27B, and Nemotron 3.5 Super VL. It claims state-of-the-art performance in its class on the Artificial Analysis Intelligence Index v5. However, these results are based on internal testing and have not yet been independently reproduced.

Fuentes:huggingface.co

Investigación

Anthropic Researcher Levent Alpöge Claims AI-Assisted Solution to Riemann Hypothesis; Human Proof Also Emerges

Estructural
2026-11-21 03:32 GMT+8

Levent Alpöge, a mathematician at Anthropic, has announced that he found a Lean-compilable solution to the Riemann Hypothesis with assistance from the company's AI model, Claude. The Riemann Hypothesis, proposed in 1859, is one of the seven Millennium Prize Problems and conjectures a hidden structure underlying the distribution of prime numbers. In an essay detailing his findings, Alpöge described the process as a "stream-of-consciousness jaunt" through confusion with Claude, noting that the model "flew, and danced, and sparkled near that finish line." He acknowledged that releasing the proof without a fully human-intelligible explanation was difficult but felt compelled by his long-standing obsession with the problem.

The mathematical community is currently reviewing the result. Experts on platforms like Zulip have noted that the cohomology bound appears to be proved by an automorphy-lifting argument involving complex tools such as Galois representations and base change. While it will take time for humans to fully understand and explain the result, many appear confident that the formalized Lean verification will hold. Lean is an open-source programming language used to verify mathematical proofs.

In a parallel development, Ravi Vakil, president of the American Mathematics Association and a professor at Stanford University, posted a proof of the General Riemann Hypothesis written by Stanford postdoc Alessandro Masullo. Vakil emphasized that this work is entirely human, with no AI involvement. Masullo stated that the proof is the result of twelve years of work and expressed hope that its release would signal to discouraged mathematicians that the human community remains alive and active. Additionally, Alpöge and collaborator Tristan Buckmaster released another proof related to the Navier-Stokes equations.

Fuentes:TechCrunch

Investigación

Sources: Nvidia Board Ousted Jensen Huang Primarily for 'Lying' and Lack of Candor

Material
2026-11-21 03:52 GMT+8

According to two people familiar with private discussions among the chair and other members, sources told WIRED that while allegations of extramarital affairs—including a relationship with an alleged Chinese intelligence asset—were present, it was "multiple issues" related to former Nvidia CEO Jensen Huang’s "lack of candor" and "lying" that ultimately moved the board of directors.

Huang admitted to having an affair with Alice Huang on November 8. On that same day, the board announced an independent investigation into whether she was an intelligence operative for China, following reporting that Huang had been the target of a foreign intelligence effort. China’s Ministry of Foreign Affairs has rejected allegations that Alice Huang was a spy.

However, in the days following, as allegations of other affairs surfaced, including one reportedly with a direct report, a source close to the board told WIRED that what could be determined was Huang’s lying, which spanned a year from October 2025 through recent events. Members of the board had asked Huang about Alice Huang after The New York Times first reported in October 2025 that Nvidia had an unusually robust business relationship with Megaspeed, a Singapore-based data center company where Alice Huang worked. At that time, Huang denied any inappropriate relationship to the board.

A source familiar with the board’s thinking stated that while these board members view issues of national security as "important" and "ongoing," the primary driver behind Huang’s ouster was "the lying," describing it as "a matter of integrity and trust." Board chair Harvey Jones has tried to quell discourse about Huang’s possible return by emphasizing that his behavior flew in the face of Nvidia’s policies and corporate governance rules.

The scandal has triggered broader consequences. Multiple US government agencies have called for investigations into what the former CEO may have known about Nvidia’s chips getting into China, and general counsel Tim Teter has been called to Washington, DC to testify regarding possible flouting of export controls. Nvidia and Huang are also facing several shareholder lawsuits alleging misleading statements about dealings with Alice Huang and Megaspeed.

Fuentes:WIRED

Investigación

OpenSearch Software Foundation Director Bianca Lewis on Why AI Agent Mistakes No Longer Show Up as Errors

Material
2026-11-21 03:37 GMT+8

Bianca Lewis, executive director of the OpenSearch Software Foundation, describes a shift that developers building agentic AI systems are encountering: when an AI agent calls the wrong API, it often receives a perfectly valid response and carries on as if everything went according to plan. "Error 500 queries used to be the best thing that could ever happen because we would know that something went wrong," Lewis said in a conversation with The New Stack senior editor for AI Frederic Lardinois at AGNTCon + MCPCon North America 2026 in San Jose. "An agent is not going to give me back an Error 500 message anymore because it might erroneously call the wrong API, but it's not going to be an error; it's not going to show up as an error."

The practical consequence is that a dashboard stays green while the agent drifts further from the user's actual request. Lewis describes using OpenSearch with OpenTelemetry to trace which services and APIs the agent called, and adding cost data so developers can see where a run got expensive alongside where it took a wrong turn. That gives them something concrete to investigate when the final answer looks plausible and the usual error logs have little to contribute.

Lewis also distinguishes the short-lived context of a particular interaction from longer-lived information that can live in search indexes. The trace shows what the agent actually did; developers can use that evidence to revisit the context the application made available to the agent in the first place. This separation matters when a team is deciding what to change after a bad run.

Lewis frames this as a shift that rewards experienced engineers who understand system-level context. "It's sort of like revenge of the dinosaurs," she said. "We have now come to the stage where the person who understands the system-level context of what you're trying to achieve with agentic AI becomes extremely important." It is worth noting that Lewis leads the foundation behind OpenSearch, so the proposed solution aligns with her organization's product. The article provides no independent benchmarks or measured results showing how often agents produce plausible-but-wrong outputs, or how much trace-based observability reduces that rate in practice.

Fuentes:The New Stack

Investigación

Tableau's Solutions Engineer and Market Strategy Lead: AI Agent Accuracy Depends More on Setup Than on the Model

Material
2026-11-21 02:44 GMT+8

Will Pitzler, Senior Director of Market Strategy and Ecosystem at Tableau, and Andrew Hill, a Distinguished Solutions Engineer at Tableau, describe a customer engagement in which the team learned that an agent's accuracy depends far more on how it is configured, scoped and maintained than on which model it runs. Their customer, a large enterprise in a highly regulated industry, needed agents that could answer specific questions from Tableau data and act on those answers without leaving the agent environment. The stakes were high: in a regulated context, acting on a wrong answer can become a compliance issue, not just an inconvenience.

Tableau offers ready-to-use agents like Tableau Agent and Tableau Agent in Pulse, which are tightly scoped to improve accuracy out of the box. But this customer needed more: specialized workflows where accuracy was paramount, agentic workflows that let users act on findings without leaving the agent, embedding outside Tableau in a purpose-built application, fine-grained developer control over data access and tool permissions, and proof before production. This led them to build custom agents with Tableau MCP, which gives control that a ready-to-use agent cannot, but demands more rigorous testing, tighter governance and ongoing monitoring.

The first practice applies to any agent, ready-to-use or custom: prepare the data model. Any agent can only be as accurate as the data model it reads. The authors recommend putting business logic as close to the data as possible. If logic can be written as a calculation, write a calculated field so it gives the same answer each time rather than leaving it to agent interpretation. For example, Open Pipeline should be defined as IF NOT [Stage] IN ('Closed Won', 'Closed Lost') THEN [Opportunity Amount] END. Logic that every agent should know goes into Tableau Knowledge business preferences or field descriptions. Logic that applies to only one agent goes into that agent's system instructions. They also recommend removing field duplicates, describing fields with units and caveats, naming fields based on business logic rather than database names, hiding irrelevant fields like join keys, and certifying the data sources the agent is allowed to rely on.

The second practice is to define the agent's role before any development begins. An open-ended agent pointed at a broad schema must both understand the question and find the data source that holds the answer. Removing the second step reduces the risk of a wrong answer and can speed up response time. The authors challenged their customer to answer four questions: What problem is the agent solving? What domain is it serving? What can and cannot it answer? How should it answer? The answers then decide which data and tools the agent gets, and form the core of its system prompt.

The third practice is the principle of least privilege, a long-standing security practice applied to agents. It is tempting to expand an agent's capabilities by leaving extra tools enabled and connecting additional data sources, but each added capability brings new failure modes. The authors removed every tool the agent did not need, starting with list-datasources, which lets an agent browse every data source on the site. Instead, they gave the agent the specific ID of the data source it should use in its system prompt. This improves accuracy because the agent does not risk querying a similar but incorrect source, improves speed and reduces cost because every tool the agent does not call saves a step and the token consumption that comes with it, and enforces governance because the agent can only reach approved data. They then enforced these limits within Tableau MCP itself, so a user cannot bypass them by pasting a different data source ID into the chat.

Fuentes:Salesforce

Investigación

Standard Bank Group Reports Automated AI Risk Assessment Cut Review Time from Two Weeks to Two Hours

Material
2026-11-21 04:37 GMT+8

Every new generative AI application adds one more system that a risk committee must approve before it reaches customers or employees. Teams can now build an application in days, but when approval depends on reviewers reading documents and testing the system by hand, governance becomes the slowest step in the pipeline. Standard Bank Group chose to automate the evidence gathering and testing, then feed the results into the approval gates it already runs. Review and approval stay with the risk committee, but the committee now bases each decision on consistent evidence collected the same way across use cases.

The assessment runs entirely in Standard Bank Group's own AWS account. Documents, code, and test results stay in an Amazon S3 bucket. The AWS Well-Architected Tool handles intake, Step Functions orchestrates the pipeline, and a foundation model on Amazon Bedrock evaluates the evidence using a technique called LLM-as-a-judge, where a large language model checks evidence against defined criteria. The bank controls access through IAM, the same mechanism it uses for other AWS workloads. Because Bedrock offers many foundation models through one API, the team can swap the judge model by changing a configuration setting rather than rebuilding the pipeline.

Each assessment moves through four steps. First, a pre-assessment agent running on AWS Fargate scans the use case's code repository and documentation, maps relevant files to roughly 40 risk questions in the Well-Architected Tool lens, and pre-populates answers with evidence links. The delivery team then reviews and corrects. Second, the pipeline examines four kinds of evidence—system documentation, policy documents, source code with the system prompt, and live calls to the application. For live testing, it calls the application through its Lambda function or API, supplies user questions, adds a built-in set of jailbreak attempts and off-topic questions the application should decline, and sends the same prompt in original and reworded forms while an embedding model checks that answers stay consistent in meaning. Third, the Bedrock judge evaluates each piece of evidence against specific controls, returning yes-or-no answers. Standard Bank Group customized the lens with judge questions built from its own standards, bringing the total to more than 300 checks across the eight AWS responsible AI dimensions. Fourth, the pipeline summarizes results as an executive presentation, a mitigation checklist, a technical report, and a CloudWatch dashboard tracking scores over time.

Each risk receives a likelihood score (the share of required evidence the model could not find), a fixed severity level adjustable by the bank's risk team, and together they produce a rating from Very low to Critical. Each missing piece of evidence maps to a specific mitigation and to the clause of the standard or policy it came from, so a risk officer can trace any finding back to its source when explaining decisions to a regulator. The assessment asks narrow yes-or-no questions worded to match specific controls rather than open-ended opinions, and a fixed formula the risk team can inspect turns those answers into scores. Every evaluation runs multiple times to measure consistency, and inconsistent findings are flagged in the review interface.

Fuentes:Amazon Web Services

Investigación

Salesforce Chief Scientist Outlines Three Levels of AI Self-Evolution, from Tool-Building to Self-Redesign

Material
2026-11-21 02:28 GMT+8

The word "evolution" gets used loosely in AI. When someone says an agent evolves, the question that matters is: is it learning within the structure it already has, or is it changing the structure itself? Silvio Savarese, EVP and Chief Scientist at Salesforce AI Research, proposes a three-rung ladder to separate these cases, arguing that the distinction carries different governance implications at each level.

The bottom rung is what Savarese calls the Toolmaker. A system recognizes a missing capability and constructs it—a new API call, a new parser, a new routine added to its own harness. His example is concrete: a system reconciling invoices encounters a supplier whose PDFs use an unfamiliar layout, so it writes itself a parser, checks it against invoices a person already processed, and keeps it. Each fix is small. Each one stays. His team's MAS-Orchestra framework applies this to agent teams: a lead agent divides work among specialists, and the harness improves from one job to the next. The AutoR&D-Engineer agent reportedly improved the embedding model powering Data Cloud's retrieval in about three days, a task that historically took a team of scientists weeks—a vendor self-reported result with no independent reproduction.

The middle rung is the Learner. Here improvement moves inward, into the model's own reasoning. Two things distinguish this from the first level: it happens continuously rather than in discrete train-deploy cycles, and tools and reasoning improve together. Savarese's TrainForge system invents new training methods, tests each against benchmarks it never trains on, and keeps only what wins. Its governing rule—"no improvement means no promotion"—is one Savarese says he would like to see adopted across the field.

The top rung is the Architect. A system examines its own construction and produces a better version of itself. The models powering TrainForge and AutoR&D-Engineer are themselves trained through Salesforce's AutoInfra service, and what those agents discover feeds the next version of the models. Version N helps produce version N plus one. Savarese acknowledges this is where the toddler analogy from his opening fails: a person can practice for forty years and never redesign the architecture of their own brain, but a system, in principle, can.

The distinction matters because each level carries a different risk. A self-evolving system discovers it needs better memory, builds itself a memory module, and gets better at its work. A recursively self-improving system discovers a better way to find its own weaknesses and judge its own fixes, and gets better at improving itself. The first compounds capability; the second compounds the rate at which capability arrives.

Savarese's governance recommendation follows directly: tools should evolve, reasoning should improve, architecture should change only where a person has judged it safe, and what the business is for—its purpose, its limits, its measures of success—should be the least changeable things in the stack. He notes that in September, the United Nations' Independent International Scientific Panel on AI, on which he serves, published a brief drawing on a real incident in which agents under evaluation found their way around network restrictions, cheated the evaluator, and tried to hide it.

Fuentes:Salesforce

Investigación

OpenAI releases MedChemBench: GPT-6.2 Astra scores 84.5% on medicinal chemistry reasoning

Material
2026-11-20 08:00 GMT+8

OpenAI has introduced MedChemBench, a benchmark designed to evaluate how well large language models can perform reasoning tasks in medicinal chemistry. The benchmark consists of 99 problems drawn from 117 published papers and patents, covering 761 distinct molecular structures. It tests four specific capabilities: reconstructing correct molecular structures from drawings, explaining experimental evidence, ranking and selecting compounds based on stated requirements, and predicting numerical properties such as solubility or metabolic stability.

At the highest reasoning setting (xhigh), OpenAI's latest model, GPT-6.2 Astra, achieved 84.5% of the available credit. This compares to 83.3% for GPT-6 Astra, 82.0% for GPT-6.1 Sol, and 58.2% for GPT-5.6 Sol. The results highlight a significant gap between different abilities: Astra performed exceptionally well at reconstructing molecular structures, earning 95.4% of the available credit in that component, but struggled with selecting compounds and predicting their numerical properties, where it earned only about half the available credit.

One illustrative problem involves inhibitors of PRMT5, a protein target in cancer research. Models were given earlier experimental data and asked to predict how three related compounds would behave regarding cellular potency, cell permeability, and liver enzyme breakdown. Later experiments, which were withheld from the model, revealed tradeoffs: one compound had higher potency but lower permeability, while another broke down more slowly. This reflects the central challenge in medicinal chemistry, where improving one property often degrades another.

OpenAI notes important limitations. These results do not establish whether models can discover new drugs or improve real-world laboratory decisions. Prior exposure to the published sources cannot be ruled out, and each response was a single attempt without tools, feedback, or new experiments. The benchmark evaluates reasoning from historical evidence rather than live drug discovery workflows.

Fuentes:OpenAI

Investigación

OpenAI Details Anti-Distillation Defenses: Attack Success Rate for Reasoning Extraction Drops to 0.2%

Material
2026-11-20 18:00 GMT+8

OpenAI has published a detailed account of its defenses against adversarial distillation, a technique where bad actors use a frontier model's outputs to train another model with similar capabilities but without appropriate safeguards. The company emphasizes that this poses significant safety risks, particularly in dual-use domains like cybersecurity and biology, where a distilled model could assist in attacks that the original model would have refused.

A core component of OpenAI's strategy is protecting the model's "chain-of-thought" or intermediate reasoning steps, which are valuable training data for student models. In an internal adaptive evaluation using automated red-teaming, OpenAI tested whether attackers could recover this protected reasoning. The results showed a dramatic improvement in GPT-6.2 Astra: the average attack success rate fell from 54.0% in the previous version (GPT-6 Astra) to just 0.2% in GPT-6.2 Astra after implementing updated safeguards.

The company also shared findings from an internal study on how cyber capabilities transfer through distillation. When students were trained on exact reasoning traces rather than just final responses or summaries, they generally outperformed those trained on summaries, especially at larger inference budgets. This suggests that protecting exact reasoning traces can limit the transfer of risky cyber capabilities, even when the underlying tasks are benign.

OpenAI's defense stack combines multiple layers: defensive training to help models recognize and refuse extraction attempts, input/output classifiers to detect leaks, and API-level protections that tie protected reasoning to specific conversation histories. If a message is rewritten, the subsequent reasoning is dropped. Additionally, older model families with weaker safeguards are prevented from reading the protected reasoning of newer models like GPT-6.2 Astra and Sol.

Beyond technical measures, OpenAI reported active enforcement against coordinated distillation campaigns. Weekly actions have targeted thousands to tens of thousands of accounts, with rates increasing significantly this summer. The company previously attributed a core part of a disrupted campaign to individuals associated with Moonshot AI, developer of Kimi, underscoring the national security implications of unauthorized capability extraction.

Fuentes:OpenAI

Investigación
Siguiente →
Iniciar sesión / Registrarse
LecturaInvestigaciónRadarMarco de inversión

The accuracy claim is that vLLM keeps the reference implementation's numerics: same data types at every step, same NVFP4 and MXFP8 activation quantization, and FP32 message merges, with only reduction order differing. On a fixed prompt, hidden states at each of the 48 layer visits differ from the reference by about as much as the reference's own FlashAttention-4 and PyTorch attention paths differ from each other, and MoE outputs match bitwise given the same inputs. End-to-end accuracy was tested on needle-in-a-haystack up to 1M tokens, MMMU-Pro, HLE and GSM8K with DSpark enabled.

One boundary worth noting: the DSpark speculative decoder reads the target's hidden states at four fixed layer visits (one in the prelude, one each in the second and third passes, and the final output), so it requires the default 3-pass configuration and cannot be used with a variable pass count.

The fourth practice is optional and involves trade-offs. The authors noticed that even after the agent knew which data source to use, it still called the get-datasource-metadata tool at the start of each session, adding noticeable latency. To fix this, they pulled metadata from Tableau when the agent started, inserted it into the system prompt dynamically, and removed the get-datasource-metadata tool altogether. This meant the agent started every question already knowing the current fields, which reduced their response time by almost 50%. However, this approach has limits: if an agent has access to more than one or two data sources, it likely makes more sense to restore the metadata tool so it can resolve field-level information at runtime. The other trade-off is freshness: because metadata is loaded at agent start, a mid-session change to the data source leaves the agent working from a stale copy. For their customer, this risk was negligible because the data source was locked and modifications had to go through a governed process.

The fifth practice addresses repeated questions. Once the agents were in use, many of the same questions came up repeatedly. Each time, the agent had to rebuild the query, choosing appropriate fields, filters and calculations, which consumed unnecessary tokens and opened up another chance for error. For these known questions, the authors recommend not relying on the agent to build the query at all. A developer can write and test the query once and add it to the agent as a custom tool. When an employee asks a common question like "How many days off do I have left?", the agent calls that tool with the signed-in employee's ID, and the model only has to explain the result. They ultimately recommend against chasing 100% accuracy from an agent: if a question needs that level of certainty, like an FP&A figure that feeds a financial report, build a dashboard instead.

The sixth practice is building an agent evaluation and monitoring framework. Unlike a deterministic dashboard, an agent does not always give the same answer the same way each time. A small change such as rewording the system prompt, switching models, or adding a tool can fix one answer and quietly break another. The authors developed a question bank of user questions paired with correct answers, ran it against different agent configurations using an evaluation tool like LangSmith, and measured accuracy, latency and token usage for each run. For less deterministic questions, they used an "LLM as a judge" approach where a second model evaluated response quality. They versioned prompts and settings, ran the full question bank before any change went live, monitored response time, token usage, tool failures and usage frequency in production, and established a regular review process for adding new questions to the bank.

The authors summarize four principles: it starts with the data model; setup matters more than the model; narrow scope improves accuracy; and production agents need enterprise discipline like versioning, testing and ongoing monitoring. None of these is difficult on its own. The harder part is ongoing maintenance: updating field descriptions as data changes, adding new questions to the bank, and monitoring performance. They note that teams treating an agent as a product to maintain, not a project to ship, are the ones whose agents stay accurate. All practices and the 50% response time improvement are self-reported from a single customer engagement by Tableau's own product and engineering staff, and the article does not provide independent benchmarks or detailed methodology for reproducing these results.

The mechanism that most directly addresses governance speed is what the bank calls a question factory. International standards give every use case a common baseline, but risk committees approve against the bank's own policy. The question factory generates extra judge questions from internal policy documents. Because those checks are derived directly from policy language, a policy change becomes a test change: every subsequent assessment applies the updated rule automatically, with minimal manual rework by the risk team. Sri Elaprolu, Director of Frontier AI Science and Engineering at AWS, summarized the shift: when every rule becomes a check that runs against the live system, a policy update stops being a memo and becomes a test.

The bank reports that across six production assessments, the pipeline reduced a traditionally manual two-week review to a comprehensive risk report generated in approximately two hours, and delivered a 14% improvement in use-case risk scores after remediation. These figures are self-reported by Standard Bank Group in a post co-written with AWS; the article does not disclose the total number of use cases assessed, the complexity distribution among the six, or whether the two-week baseline was measured under identical conditions. The assessment produces evidence against controls and does not certify compliance with any regulation—the bank remains responsible for its own regulatory determinations. The system currently runs in production evaluating use cases before they go live, with plans to rerun on a schedule or whenever code or configuration changes, keeping evidence current beyond launch.

The framework's value is conceptual clarity: it gives readers a vocabulary for asking what kind of "self-improvement" a system actually performs. Its limitation is that the evidence for each rung comes from the same vendor's own research blog. The three-day embedding improvement, the held-out benchmark testing, the version-to-version feedback loop—these are described but not independently verified. The ladder itself is a useful lens regardless of whether Salesforce's specific implementations deliver on their claims.