読書リサーチレーダー投資フレームワーク
ログイン / 新規登録
すべて技術製品投資株式市場その他

読書

2026-11-0733 投稿

更新

Colleen Aubrey, Head of Applied AI at AWS: Don't Invent New Metrics for AI, Measure It with Your Existing Business Metrics

トピック · AWS AI指标之争重大
2026-11-07 02:56 GMT+8

Colleen Aubrey has worked at Amazon for over twenty years, having built customer-facing products for millions of sellers and advertisers. In 2026, in her capacity as Senior Vice President of Applied AI Solutions at Amazon Web Services (AWS), she offered a rather direct suggestion for corporate AI practices: stop inventing new AI metrics.

Her core argument is that measuring whether AI is successful does not require a completely new system of metrics.

"I worry that people start trying to come up with new metrics to measure the impact of AI, but it’s the same metrics," she explained. "It’s your business metrics that matter."

This statement sounds simple, but it points to a widespread practical issue: after investing in AI, many companies build AI-specific dashboards for token consumption, prompt usage frequency, and model call tracking, only to find that while these numbers rise, business results remain stagnant.

The proof point Aubrey provides is Wells Vehicle Electronics. According to AWS reports, the company reduced inventory by 7% within three months while expanding the number of SKUs actively monitored from 20 to 2,000. She emphasized that no one at Wells talks about tokens; they talk about working capital. If a project cannot be tied to a metric the Chief Financial Officer is already tracking, it is an experiment, not an investment.

What does this distinction mean in practice? Aubrey suggests that before any AI project receives funding, bind it to a business metric the company is already tracking—inventory levels, problem resolution rates, hiring cycle time, production cycle time—and set a baseline. If this metric does not change within a defined time window, the project should not scale, regardless of how impressive the demonstration was.

Her argument also touches on a deeper mechanism: AI usually does not eliminate bottlenecks but moves them. Internal experience at Amazon shows that when AI coding tools were widely deployed, teams first discovered insufficient test coverage; after fixing tests, they found pipeline issues; after fixing pipelines, they found manual approval steps slowing down deployment. The result for Aubrey's team this year was a doubling of the rate of pushes to production, but she explicitly stated that this speed increase did not come from the coding assistant itself, but from fixing all the downstream problems exposed by the assistant.

The implication for enterprise procurement of AI tools is this: if you buy the tool but skip pipeline repair, activity will rise but output will remain flat. She recommends reserving budget and personnel for downstream fixes rather than discovering new constraints six months later.

Aubrey also proposed a lesson at the product strategy level: feature parity is a trap. When she took over the AWS supply chain product, the team was building five modules attempting to catch up with ERP vendors that had decades of head start. When she rebooted the product as Connect Decisions, she asked a different question: where is the pain most acute, and where are existing tools weakest? The answer was planners—who spend their days firefighting, overwhelmed by a flood of signals, by the time they make a decision the moment has passed. This principle of picking a wedge rather than a platform is what she sees as key to moving enterprise AI projects out of pilot mode. A narrow, painful problem yields quick wins, clear metrics, and a credible foundation for expansion. Projects that attempt to reimagine an entire function at once often stall.

ソース:SiliconANGLE

リサーチ

Commentary on the German Digital Summit: AI Sovereignty Goals Escalate Year After Year, Yet Binding Timetables Never Appear

トピック · 德国AI主权争议重大
2026-11-07 18:05 GMT+8

The narrative of AI sovereignty within the German Federal Government follows a predictable cycle: grand goals are announced at summits, partnerships stall a year later, actual procurement shifts to American companies, and then a new summit announces even larger goals. Following the 2026 Digital Summit, t3n commentator Marcel Romahn pointed out that the core problem with this cycle is not that the goals are insufficiently high, but rather the lack of binding timetables and execution speed.

In November 2025, the German Federal Government announced a massive AI offensive at the German-French Digital Sovereignty Summit, establishing concrete cooperation with French AI company Mistral and planning a framework contract with software enterprise SAP. The goal was to reduce dependence on US tech giants and become a self-sufficient technology provider. A year later, the Mistral partnership was shelved and blurred into a broader intent, while the SAP framework contract was never signed. Consequently, the resulting vision for autonomous AI administrative software and agents was downgraded. The German Federal Ministry for Digital Affairs stated it remains committed to the cooperation, but its wording was noticeably more conservative.

Meanwhile, actual infrastructure decisions deepened reliance. The German Federal Intelligence Service (BND), the Military Counterintelligence Service (MAD), and the Federal Office for the Protection of the Constitution planned to use the Google Cloud platform. Nvidia was actively courted and received a strategic letter of intent. The Bavarian State Government had previously announced an investment of billions of euros for Microsoft's administrative software as a US hyperscaler, though this was withdrawn due to internal disputes and public protests. Nevertheless, data protection experts and observers believe Germany as a whole has not moved toward security or sovereignty.

The 2026 Summit launched an 18-point action plan aimed at doubling general computing capacity by 2030 and ideally quadrupling AI computing capacity, while continuing to support startups and research institutions. Romahn acknowledges that the challenge is indeed immense—developing autonomous technology while maintaining economic competitiveness is difficult, there are no viable alternatives to chip and raw material dependencies, and a purely European path appears naive. However, he points out a key flaw: the plan sets no binding timetables for most measures, nor does it establish action nodes for individual agencies. Four years of global AI development pass in the blink of an eye; without speed constraints, grand goals are merely another declaration.

The author's conclusion is clear: Germany needs not more spectacular summit commitments, but achievable milestones, greater speed, and enforceability. Otherwise, Germany will continue to remain in a customer position, with most goods on the shelf still 'Made in America.' This judgment itself is an attribution argument rather than a verified prediction—it is based on specific factual gaps between two summits, but the results four years from now still depend on execution.

ソース:t3n.de

リサーチ

Security firm reports bypassing ChatGPT 'Lockdown Mode' to access chat history; OpenAI patches flaw

重大
2026-11-07 00:05 GMT+8

Researchers from US cybersecurity firm Check Point Research (CPR) discovered a method to bypass OpenAI ChatGPT's 'Lockdown Mode', a feature designed to protect users from prompt injection attacks by restricting external connections and disabling certain functions. The researchers exploited a flaw in the nested sandboxed frames used for HTML previews within code blocks, allowing popups to escape sandbox restrictions and potentially access the entire chat history.

'Lockdown Mode' was originally designed to reduce the risk of sensitive data exposure via prompt injection by limiting features like live web browsing, Deep Research, Agent Mode, and file downloads. However, CPR noted that while ChatGPT intercepted direct attempts to open new windows, the underlying sandbox permissions still allowed popups to exist and escape. By creating nested frames, the researchers could access unmodified popup functions, bypassing all restrictions placed on the original HTML preview.

This vulnerability could lead not only to data theft but also to malware deployment, phishing attacks, or credential theft. The most dangerous scenario described was a 'zero-click' attack, where the popup appears without user interaction. CPR emphasized that 'AI security controls are only as strong as the wider application environment around them,' indicating that model-level security measures alone are insufficient and enterprises must consider the overall attack surface comprising applications, runtimes, and connectors.

OpenAI implemented additional safeguards server-side after receiving the disclosure to disable this workaround. Although the vulnerability has been patched, the incident serves as a reminder for users and organizations to remain vigilant about application-layer security boundaries when handling sensitive information with AI tools and to seek greater visibility into AI usage.

ソース:TechRadar

リサーチ

US Adds 165,000 Jobs in October, Far Exceeding Expectations; Fed Hike Odds Jump

トピック · 美国10月非农就业数据重大
2026-11-06 23:53 GMT+8

The Bureau of Labor Statistics reported that the US economy added 165,000 jobs in October, far exceeding economists' expectations, while the unemployment rate ticked down to 4.1%. The agency also revised August and September figures upwards, indicating that the economy is hotter than previously anticipated.

This resilience adds to evidence that the US Federal Reserve will hike interest rates at its December meeting. According to the CME FedWatch tool, odds of a rate hike jumped 12 percentage points following the release. Markets interpret this as the central bank focusing on tackling elevated inflation stemming from the Iran war.

However, some traders questioned whether the Fed would have the appetite to raise rates following Thursday’s $2.5 trillion tech selloff, sparked by fears that Democratic gains in the US midterms could slow the AI buildout. “Will the Fed want to raise interest rates in the face of a market which is teetering?” an equity trader asked.

ソース:semafor.com

リサーチ

German Police Use AI-Generated Image to Hunt Leipzig Drone Suspect

トピック · 莱比锡无人机袭击案クイックテイク
検証済み 2026-11-07 17:33 GMT+8

The German Federal Criminal Police Office (BKA) and the Federal Prosecutor's Office published an AI-generated image on Friday, November 6, to help identify a third suspect in the attempted drone attack at Leipzig Airport. The image depicts a slender person wearing a black hooded jacket and carrying a large backpack while riding an e-bike.

According to a report by Die Welt, investigators stated that the unidentified individual may have been near the airport between July 31 and August 4. Despite the summer heat, the person was unusually dressed for the season, wearing long sleeves, gloves, and a cap pulled low over their face. Two suspects have already been identified, and this composite image is intended to track down a potential third accomplice linked to the incident.

The attempted attack occurred on the evening of August 4 when an airport employee discovered a drone loaded with explosives on the ground near Ukrainian cargo planes. Leipzig Airport serves as a key hub for military aid deliveries to Ukraine. The German government previously blamed Russia for orchestrating the plot, which Moscow has denied.

ソース:DIE WELT

リサーチ

Alibaba Cloud Engineering Team: Code Written Ten Times Faster, Why Delivery Cycle Only Shortened by 1.8 Days

トピック · 阿里AI工程交付瓶颈重大
検証済み 2026-11-07 16:07 GMT+8

For a complex feature, the typical team requires about 20 working days from requirement proposal to launch, with coding occupying only two days. Assuming an AI coding tool increases this speed tenfold, reducing two days to 0.2 days, the entire delivery cycle shortens from 20 days to just 18.2 days. In practice records published by the Alibaba Cloud engineering team in November 2026, this arithmetic places the issue on the table: accelerating code writing is not the bottleneck; rather, requirement discussions, solution confirmation, testing and verification, cross-module integration, and release preparation are the actual constraints. Once code is written faster, these preceding and following steps become even more conspicuous.

Their approach involves breaking down the work undertaken by different roles in the R&D process into 15 distinct Agents, which a Leader schedules according to task requirements. Each Agent possesses independent skills, tools, and context, with clear responsibilities: the Researcher understands requirements and the codebase while building a problem evidence graph; the Explorer identifies isomorphic implementations to form candidate solutions; the Spec structures goals, boundaries, and acceptance criteria; the Code Worker implements candidate solutions in independent branches and sessions; Test/Harness executes compilation, testing, and end-to-end verification; Judge/Verify independently selects the best candidate, performs semantic acceptance, and conducts adversarial review; and Deploy prepares the environment and executes release verification.

Two design choices within this architecture are considered most critical by the authors. First, implementation and verification must be completed by different Agents, ensuring that verification maintains an independent perspective—if the Agent writing the code also judges its own correctness, verification loses its meaning. Second, context across iterations must be preserved—subsequent Agents must be able to continue using information regarding what the previous Agent did, why it was done that way, and what was discovered during verification. If tests fail, the process returns to the original Code Session for continued fixes; if the solution direction is incorrect, Research and Explore are redone with failure evidence; if repeated attempts still do not converge, a Human Gate is triggered to request human intervention for judgment.

Beyond AgentCore, which handles execution, they established AgentLoop as an optimization plane. AgentLoop records the task completion process: which models and tools were called, where failures occurred, how many rounds of rework took place, and whether the final result met requirements. These Trace, Session, and Trajectory data undergo processing through Pipeline, Dataset, Evaluation, and Experiment stages, transforming into reproducible samples, comparable metrics, and repeatedly verifiable experiments. Optimization occurs in three ways: manual, where domain experts define good results, metrics, and evaluation Rubrics; semi-automatic, where the system generates optimization suggestions (identifying tasks consuming excessive Tokens, inefficient steps, or tools consistently reporting errors); and automatic, where experience repositories extract insights from Traces using the RAC method and dynamically inject them into Agents.

ソース:Alibaba Cloud Community

リサーチ

US AI Voice Firm SoundHound Reports Record Q3 Revenue but Cuts 27% of Workforce, Guides for EBITDA Profitability in 2H 2027

重大
検証済み 2026-11-07 16:07 GMT+8

SoundHound AI, Inc., a US-listed provider of voice and agentic AI technology, filed an 8-K report on November 5, 2026, disclosing financial results for the third quarter ended September 30, 2026, alongside a significant restructuring plan. While the company reported record quarterly revenue, it also announced a reduction of approximately 27% of its global workforce to control costs, setting a target for adjusted EBITDA profitability in the second half of 2027.

Financial data shows SoundHound AI's Q3 revenue reached $62.4 million, a 48.5% year-over-year increase. However, GAAP gross margin declined to 34.1% from 42.6% in the same period last year, and non-GAAP gross margin fell to 50.3% from 59.3%. The GAAP net loss narrowed significantly to $46.4 million from $109.3 million in the prior year; the non-GAAP net loss was $21.2 million. Adjusted EBITDA loss stood at $20.7 million. As of the end of the quarter, the company held $207.6 million in cash and cash equivalents.

As part of the restructuring plan implemented on November 4, SoundHound AI aims to align teams and resources with strategic priorities, including cutting about 27% of its global staff. The company estimates pre-tax charges of $18 million to $22 million, substantially related to severance payments and post-employment benefits, with expected cash expenditures of $16 million to $19 million. These charges will be primarily recognized in the fourth quarter of 2026. Management stated that savings from the restructuring will be used to further strategic initiatives.

In its business outlook, SoundHound AI identified $120 million in annualized cost synergies, with over $80 million in actions scheduled to take effect before January 1, 2027. The company projects full-year 2026 revenue between $250 million and $260 million, and 2027 revenue between $350 million and $400 million. CFO John Collins emphasized that integrating the LivePerson acquisition, exiting low-margin businesses, and shifting to a hosted-service model would improve revenue stability and drive sustainable profitable growth.

ソース:sec.gov

リサーチ

Optical transceiver maker Zhongji Innolight plans to cancel 5.65 million repurchased shares; net profit for first three quarters surges 211.81%

トピック · 中际旭创AI光模块业务重大
検証済み 2026-11-07 15:59 GMT+8

Chinese optical transceiver manufacturer Zhongji Innolight (300308.SZ) released a voluntary announcement on November 6, 2026, stating that it will change the purpose of its previously repurchased A-shares from "implementing employee stock ownership plans or equity incentive plans" entirely to "cancellation and reduction of registered capital." The proposal was approved by the board on October 29, 2026, and is pending review at an extraordinary shareholders' meeting.

According to the disclosure, the company cumulatively repurchased 5,653,063 shares between September 1 and September 23, 2026, accounting for 0.48% of its total share capital, with a total payment of approximately RMB 4.997 billion (excluding transaction fees). Upon completion of this cancellation, the company's total share capital will decrease accordingly, aiming to further enhance financial metrics such as earnings per share (EPS) and net assets per share. Additionally, the company noted that Chinese regulators recently lowered the interest rate for share buyback and holding re-lending facilities to 1%, indicating it would utilize policy tools to maintain corporate value.

On the operational front, benefiting from high industry prosperity and strong capital expenditure from major customers, Zhongji Innolight reported operating revenue of RMB 69.547 billion for the first three quarters of 2026, a year-on-year increase of 178.13%. Net profit attributable to shareholders reached RMB 22.238 billion, up 211.81% year-on-year. Orders for 800G and 1.6T high-end products in the third quarter achieved good sequential growth, with accelerating revenue from 1.6T products serving as a key driver.

The company stated that current orders from major customers have largely covered the entire year of 2027, and some key clients have provided clear demand guidance for 2028. As demand for next-generation optical interconnect products like NPO and XPO is expected to gradually release in the second half of next year, the company is actively preparing materials and capacity to meet future growth.

ソース:static.cninfo.com.cn

リサーチ

AMD Releases InferaSim: Predicting LLM Serving Performance with Sparse Measurements to Cut Tuning Costs

トピック · InferaSim重大
2026-11-06 08:00 GMT+8

US semiconductor company AMD released the open-source tool ROCm InferaSim on November 6, 2026, aiming to address the prohibitive costs of tuning Large Language Model (LLM) inference service configurations. Before deploying an LLM service, engineers must select parameters such as the inference engine, parallel layout, precision, and scheduler settings, generating thousands of candidate configurations. Traditional methods require hardware measurement for each configuration, with each trial tying up multi-GPU nodes for minutes. AMD stated that measuring every candidate in their search space would take approximately 25,000 GPU-hours, leading most teams to try only a few guesses and keep the best, while missed guesses result in extra GPU waste or failed latency targets.

The core idea behind InferaSim is "measure sparsely, transfer analytically." The tool separates parameters that change which kernels run (such as model, dtype, attention backend, graph mode, and speculative decoding) from those whose costs scale analytically (such as batch size, concurrency, context length, tensor-parallel width, and replica count). For the former, InferaSim performs one short measured run, called an "anchor," saved as a JSON file containing a batch sweep of measured prefill and decode step latencies plus a signature of what was running. For the latter, the tool performs fast analytical projections based on the anchor. Each projection takes under half a second on a single CPU core.

Internal tests on AMD's own Instinct MI355X GPUs showed that 11 anchors cost 9.1 GPU-hours and covered 226 configurations that took 204 GPU-hours to measure fully. This means GPU time is spent confirming a shortlist rather than exploring all possibilities. InferaSim can predict whether a recipe fits in HBM, the latency distribution under a given arrival process, the number of replicas needed for an SLO, and the best split between prefill and decode before any deployment exists.

The tool also introduces a discrete-event simulator (DES) to simulate Poisson arrivals, providing latency percentiles and queue wait times instead of just averages. Additionally, InferaSim plots the throughput-versus-interactivity Pareto frontier, helping teams find the configuration with the highest throughput per GPU at a given interactivity level. According to AMD's published paper and blog, InferaSim's predicted frontier kept all 10 measured-frontier configurations for GLM-5.2 and 11 of 12 for MiniMax-M3.

Developers can access the code via GitHub and install it on hosts supporting the ROCm software stack. InferaSim offers two modes: the default "anchored mode" uses matching anchors to calibrate the step model for absolute number predictions, while "simulate mode" is fully analytical and requires no anchor, suitable only for ranking candidates within one engine. This tool provides AI infrastructure teams with a more economical and efficient path for optimizing inference services.

ソース:ROCm Blogs

リサーチ

Google Announces First Wyoming Data Center with $17 Billion Investment, Featuring Fuel Cells and Carbon Capture

トピック · 谷歌怀俄明数据中心重大
2026-11-06 01:00 GMT+8

On November 5, 2026, Google announced plans to build its first data center in the state of Wyoming, representing a long-term investment of $17 billion. Developed in partnership with Tallgrass Energy, Black Hills Energy, American Electric Power (AEP), and Related Digital, the facility aims to deliver lasting economic benefits to families and businesses in Laramie County.

To address the power supply challenges facing AI infrastructure, the project employs an innovative energy architecture. Google has contracted with AEP to feature a "first-of-its-kind," commercial-scale 702-megawatt (MW) fuel cell development equipped with carbon capture and storage (CCS). These fuel cells, manufactured in America by Bloom Energy, efficiently convert natural gas into electricity through electrochemical reactions, minimizing emissions and water usage.

Regarding carbon management, the project will utilize Tallgrass’s nearby "Eastern Wyoming Sequestration Hub" for the permanent geologic storage of a large majority of its carbon emissions. This builds upon Google’s first US CCS project announced last fall, bringing its total contracted CCS capacity to nearly 1 gigawatt (GW). Additionally, Google is sourcing power from gas generators operated by Tallgrass Energy and Black Hills Energy, and conducting a technical front-end engineering and design (FEED) study to evaluate fitting Tallgrass’s combined cycle turbines with CCS technology.

Beyond energy innovation, Google emphasized responsibility to the local community. The data center will be air-cooled to minimize strain on local water resources. Google committed to paying for 100% of the power it uses and covering infrastructure needs directly driven by its growth. The project is expected to generate hundreds of millions of dollars in local property taxes for Laramie County and School District 1. Google and its partners have also established a Community Council to address the impacts of data center growth on Cheyenne, supporting K-12 education, workforce development, and skilled trades training across the state.

ソース:Google

リサーチ

Salesforce Architect Proposes Closed-Loop Testing Method: Using Immutable Snapshots to Prevent AI-Generated Code from Exhausting Transaction Budgets

重大
2026-11-06 02:54 GMT+8

When generating code on the Salesforce platform, AI agents face a problem not prominent in traditional development: they optimize for the immediate task rather than the overall resource budget of the transaction in which that task executes. Even if each feature passes tests individually, cumulative consumption may still breach platform hard limits during a seemingly harmless change.

In an article published on November 5, 2026, Sakthi Kuppusamy, Senior Technical Architect at Salesforce, describes this compounding pattern: Sprint 1 adds account validation logic, consuming 15% of the SOQL query limit; Sprint 2 adds data enrichment, rising to 32%; Sprint 3 adds related contact lookups, reaching 51%; Sprint 4 adds a duplicate check worth only 8% on its own, totaling 59%. No single change is problematic in isolation, but in traditional development, such accumulation unfolds over months, giving teams time to notice degradation, whereas agents can compress this timeline into a single afternoon of merged pull requests.

Kuppusamy's core argument is that governance limits are not platform constraints to be circumvented, but architectural contracts to be designed for from the start. To enforce this contract, he proposes a three-layer mechanism.

The first layer is the immutable snapshot class. Deploying a test utility class named GovernorLimitSnapshot (annotated @IsTest, so it does not count against the organization's Apex code size limit), the agent must consume this tool without modifying or redefining it. This class captures resource allocation and consumption before and after execution, centralizing threshold logic in one place to prevent agents from scattering assertions throughout the codebase.

The second layer is budget allocation per execution context. Synchronous transactions have a SOQL query limit of 100 and CPU time of 10,000 milliseconds; asynchronous transactions allow 200 queries and 60,000 milliseconds. However, different execution contexts should be allocated different proportions: Triggers (shared events) are recommended at 40% to 50%, because other triggers, flows, and validation rules exist within the same transaction; Service classes (single responsibility) at 60% to 70%; Queueable, Future, and Batch Apex (isolated contexts) at 80% to 85%; Scheduled Apex (lightweight orchestration) at 20% to 30%. These budgets are not additive—a service class called from a trigger draws from the trigger's budget and has no independent quota of its own.

The third layer is closed-loop feedback in CI/CD. When a test violates a threshold, the pipeline does not simply mark the build as failed; instead, it sends a structured prompt to the agent containing violation details, budget thresholds, and optimization hints. After the agent refactors and resubmits, if the second CI run still exceeds limits, the feedback includes the delta between attempts: what improved and which metric remains over budget. If the third push still fails, the pipeline halts the loop, blocks the merge, and opens a manual architect review task with the full run history. This three-strike boundary routes fundamental architectural issues (such as a feature that genuinely cannot fit into a synchronous transaction) to human decision-makers to determine whether to migrate logic to an asynchronous Queueable or adjust the transaction budget.

ソース:Salesforce

リサーチ

Datadog Troubleshooting Prompt Cache Misses: Four Silent Code-Level Traps

トピック · Datadog重大
2026-11-06 08:00 GMT+8

The AI applications team at Datadog encountered a counterintuitive phenomenon while investigating LLM costs: multiple internal applications had enabled prompt caching options, yet there were zero cache hits in actual requests. Each request contained over 180,000 input tokens, with single-request costs ranging from $0.74 to $0.82. Once caching took effect, the cost for the same task could drop to between $0.02 and $0.14. The gap was so significant that finding the cause seemed impossible.

Staff Engineer Bruno Volpato and his team recorded the complete troubleshooting process. Their methodology was to first use Cloud Cost Management to view overall spending, then use Agent Observability to trace down to individual model calls, and finally compare the differences between the first call and subsequent calls within the same task. An agent might call the model multiple times to complete one task, repeating system instructions, tool definitions, and conversation history each time. If caching worked correctly, subsequent calls should read these repeated contents from the cache.

The investigation revealed four categories of silent failure modes. The first category involved mutable content mixed into reusable prefixes. A chat service concatenated system instructions with current time, organization details, feature flags, and incident context. Prompt caching relies on byte-by-byte matching of request prefixes—if a timestamp appears before the instructions, even if the subsequent instructions are identical, the cache cannot hit. Their fix was to place stable instructions first and request-specific details last, using cache boundary markers to denote the end of the reusable portion where supported.

The second category was non-deterministic ordering of tool lists. That chat service built its tool list in Go, and Go's map iteration does not guarantee order. One request might send tools in the order A, B, C, while the next sends them as B, A, C. Since tool descriptions are part of the cached prefix, a change in sequence breaks the match. The patch was simple: sort by name before building the list. The team also added a test that inserts the same tools into maps in different orders to verify the resulting list is consistent.

The third category was caching options not being passed to the actual request. One application enabled an option called WithPromptCaching, but inspection of requests sent to Anthropic revealed that this option only added a beta header; the request body lacked the control parameters telling the provider what to cache. Another application was more direct: the SDK's caching option was simply not passed to the component preparing the provider request.

The fourth category was API routes that did not support caching. Datadog's gateway can route through Anthropic's native Messages API or a compatibility endpoint formatted for OpenAI. The native route supports cache reads, while the compatibility route has none. Anthropic's documentation confirms that the compatibility endpoint does not support prompt caching. Migrating applications to the native API became a potential fix path, but the team needed to verify that message roles, tool calling, and streaming responses still functioned correctly under the new route before recommending it.

ソース:Datadog

リサーチ
次へ →
ログイン / 新規登録
読書リサーチレーダー投資フレームワーク

It must be noted that the data for Wells Vehicle Electronics comes from AWS's retelling and has not been independently verified. Aubrey's own narrative also indicates that actual productivity gains from AI at Amazon largely came from infrastructure repairs rather than AI tools themselves, making it complex to attribute results solely to AI investment as a single variable. Her advice serves better as an evaluation discipline—using existing metrics to verify whether a project actually changes business outcomes—rather than as causal proof.

In terms of practical effects, validation was conducted in a constrained scenario involving the Trace to Trajectory service, which converts Trace data into trajectory data and frequently encounters new parsing exceptions in production. They selected 19 issues: approximately 10 produced implementations roughly equivalent to those of human experts, while another 5 approached expert level, compressing the delivery cycle from about one month to under a week. In offline experiments, they selected 20 high-quality human-expert Commits from the codebase, reverse-generated requirements to build a Benchmark, and had the Agent team complete these tasks, comparing their results item-by-item against expert implementations.

The authors frankly listed four categories of persistent issues: requirements may be clearly stated, but architectural constraints might not be transmitted effectively (for example, when aiming to extend existing API capabilities, an Agent might choose to add a new API, which is easier to complete for the isolated task but fails to meet the team's design requirements at the global system level); real-world scenarios always bring unforeseen problems (AI application inputs are far more open than traditional software); the same problem may recur (as code, business rules, and runtime environments change, experience must be continuously accumulated to avoid repeating pitfalls); and fixing one problem may impact other tasks (modifying a Skill may pass current use cases but cause regression in others). The existence of these issues indicates that multi-Agent orchestration does not run automatically once the framework is built; it requires continuous observation, adjustment, and accumulation.

Kuppusamy also highlights the necessity of defending against agent circumvention: an agent stuck in a feedback loop will seek the cheapest path to a green build, potentially shrinking test data arrays, commenting out assertions, swallowing errors with try-catch blocks, or directly editing thresholds. The countermeasure is to restrict the agent's write permissions to implementation source code only, rendering the snapshot tools and threshold metadata records as immutable infrastructure.

The boundaries of this method are also clear. Some limits resist meaningful unit testing: callouts require mocks that do not reflect real latency, platform event publish limits behave differently under test isolation, and asynchronous job limits are difficult to simulate at scale. These require supplementation with production monitoring. Sandbox tests cannot reproduce index contention and row locking under production concurrency, meaning thresholds during testing may underestimate actual consumption. Overly aggressive thresholds can cause build failures due to false positives rather than real logic errors; it is recommended to start generous, tighten based on historical data, and distinguish between warning tiers and violation tiers.

Volpato emphasized a troubleshooting trap in the article: you cannot rely solely on aggregate cache hit rates. They found that applications and gateways might both record the same model call, leading to double-counting of costs; conversely, if records lack cache fields, caching appears unused. Some records omit system instructions or tool definitions, making it impossible to judge why the provider didn't reuse previous inputs. When evaluating fix opportunities, the team also pointed out an easily overlooked cost dimension: some providers charge extra fees for cache writes. If you repeatedly write content that is rarely reused, costs may actually increase.

The value of this article lies not in discovering a brand-new caching principle—the mechanisms of prefix matching and tool order consistency are not new discoveries. Its contribution is demonstrating a real troubleshooting process: starting from aggregate cost anomalies, narrowing down to discrepancies at the individual request level, and ultimately pinpointing four specific code-level causes. For teams running LLM applications with poor caching results, this troubleshooting path and these four patterns are far more directly usable than any generic principles.