読書リサーチレーダー投資フレームワーク
ログイン / 新規登録
すべて技術製品投資株式市場その他

読書

2026-11-20117 投稿

更新

Google Quantum AI Validates Quantum Contextuality on 105-Qubit Processor, Establishing New Hardware Benchmark

重大
2026-11-20 04:03 GMT+8

Researchers at Google Quantum AI, the research division of US tech giant Google, have published a study in Science demonstrating and measuring "quantum contextuality" on their Willow superconducting quantum processors. Previously observed mainly in small systems of a few particles, this fundamental quantum property has now been validated in systems spanning up to 105 qubits, marking a significant step in understanding its role as a computational resource.

The team used the "magic square game" to illustrate contextuality. In this non-local game, two players, Alice and Bob, must fill in rows and columns without communicating, winning if their intersecting values match. While the best possible classical strategy wins only 88.89% of the time, quantum entanglement allows for a theoretical 100% win rate. On the Willow processor, the qubits achieved a 98.30% win rate, decisively beating the classical bound and confirming that measurement outcomes depend on the context of other compatible measurements rather than pre-existing values.

To quantify the degree of contextuality, the researchers employed a Bell–Kochen–Specker inequality. Classical physics dictates that the correlation parameter χ must be less than or equal to 4, whereas quantum mechanics permits values up to 6. The experiment yielded χ = 5.590 ± 0.008, comfortably exceeding the classical limit. Crucially, this test is state-independent, meaning it measures an intrinsic property of the machine itself rather than the outcome of a specifically prepared input, making it a robust diagnostic tool.

Beyond the two-player game, the study also tested contextuality in multi-player GHZ parity games involving up to 71 qubits. Although noise reduces the win rate as system size increases, the quantum strategy consistently outperformed the best classical approach. The authors note that these tasks serve as a highly sensitive new benchmark for quantum hardware, capable of catching subtle errors that standard random-circuit benchmarks might miss.

ソース:research.google

リサーチ

Anthropic’s IPO Wealth Could Channel $156 Billion into Philanthropy via Public Benefit Fund and Employee Matches

重大
2026-11-20 03:10 GMT+8

According to TechCrunch, the massive wealth generated by frontier AI lab Anthropic’s Initial Public Offering (IPO) is set to be converted into unprecedented charitable funding through complex equity allocation and matching mechanisms. Based on the current trading price of approximately $95 per share, the combined value of Anthropic’s “Public Benefit Fund” (PBF) and employee share pledges with company matching could reach $156 billion. This figure represents roughly one-quarter of the total charitable donations made in the United States in 2025.

The funds originate from two primary channels. First, the company-level PBF has been allotted 5% of the company’s shares (approximately 850 million shares). At the current stock price, this creates an initial endowment of about $80 billion. Additionally, if Anthropic’s valuation grows more than 50% above the IPO price of $83, the PBF will receive further 2% increments. The PBF’s money is dedicated to research and activities aimed at increasing the benefit of artificial intelligence to humanity. Its first commitment was $200 million to study the impact of AI on workers, with future plans potentially including direct cash distributions.

Second, there is the employee donation matching mechanism. Anthropic employees have pledged shares to charitable causes of their choice, with the company matching these donations at a 3-to-1 ratio for longer-tenured employees and 1-to-1 for newer ones. Disclosures in the S-1 filing indicate that the company reserved more than 550 million shares to fulfill pledges made before September 30. Working backwards from this figure, it is estimated that employees have pledged $23 billion worth of donations, while the company’s match would add approximately $53 billion, depending on the stock price at the time of delivery.

It is important to note that not all funds come from the PBF. For instance, the $50 million donated by Anthropic to the city of San Francisco last week did not come from the PBF, according to a company spokesperson. Regarding recipients, interviews suggest that employees are most likely to support charities with effective altruism (EA) leanings, such as Longview Philanthropy, GiveWell, and Coefficient Giving. Given the widespread concerns among Anthropic employees about AI safety, significant funding is expected to be allocated to AI safety research.

While the potential amount is enormous, the actual funds disbursed will depend on whether employees fulfill their pledges and the market price of the stock when the shares are delivered. Furthermore, some shares may go into donor-advised funds, which could limit their immediate impact.

ソース:TechCrunch

リサーチ

Three Independent Data Sets Point to the Same Conclusion: AI Token Usage Hits Record Highs, but Revenue Growth Is Slowing

重大
2026-11-20 03:28 GMT+8

US companies are using AI models with unprecedented intensity while paying less for that usage. This is not an isolated finding from a single data source, but a structural trend pointed to by three independent measurement systems.

The Ramp AI Index, based on transaction data from more than 85,000 US companies, records a key turning point: the unit price of tokens fell by 21.2% over six weeks, while token usage rose to a record high. Specifically, for the week ending November 15, token spending fell 6.3% from its peak two weeks prior; however, in a month-over-month comparison (the week ending November 15 versus the week ending October 18), total spending still grew by 12%, while usage grew by 27.2%. The gap between these two figures is widening.

The mechanism behind this divergence is price competition. The average price per million tokens for OpenAI and Anthropic has dropped to $0.45, with OpenAI at $0.38 and Anthropic at $0.50, a decrease of 11.2% from the previous month. The entry of Chinese suppliers has further intensified this competitive pressure. The revenue generated by increased usage is being offset, and even reversed, by the decline in unit prices.

Data from the API routing platform OpenRouter presents a similar pattern. According to analysis published by Peter Walker, weekly token usage for closed-source models from OpenAI, Anthropic, and Google grew by 1,215% since the beginning of 2026, but weekly spending grew only 455%. The spending peak occurred during the week of October 5, after which it continued to decline. The disparity in growth rates between the two figures remains significant.

Payment processor Stripe’s AI Index provides a third independent verification from the supply side. Over the 52 weeks ending November 8, revenue for AI companies grew by 103%, but median companies grew only 41%, indicating that a few large suppliers are skewing the overall numbers. More noteworthy is the short-term trend: annualized over 13 weeks, the growth rate was approximately 80%, down from 140% just a few months ago. The slowdown is clear.

These three data sets come from completely different measurement systems—Ramp tracks corporate spending, OpenRouter measures API routing traffic, and Stripe records receipts for AI companies—yet they depict the same picture: AI is still growing rapidly, but momentum is cooling. At Ramp, this manifests as falling prices and concentrated demand; at Stripe, it shows up as slowing revenue growth among providers.

This finding has direct implications for assessing the profitability prospects of AI companies. If token prices continue to fall and usage growth fails to fill the revenue gap, then the current business model, which relies on high growth to support valuations and returns on infrastructure investment, faces pressure. The AI industry still depends on extremely strong growth to recoup investments in large-scale computing capacity construction, and the weakening of growth momentum is compressing this space.

A necessary boundary condition is that neither Ramp nor OpenRouter is fully representative of the entire market. OpenRouter, in particular, reflects only the behavior of a specific user group. The huge gap between the mean and median in Stripe’s data also suggests that overall trends may be dominated by a few top-tier companies. While the direction pointed to by these data sets is credible, the precise magnitude cannot be extrapolated to the entire AI industry.

ソース:The Decoder

リサーチ

AI Planning Tool ManiFast Secures Investment from US Accelerator Alchemist and University of Chicago, Amount Undisclosed

クイックテイク
2026-11-19 11:05 GMT+8

According to Platum, ManiFast, a Korean startup operating an AI-driven planning workspace, announced that it has secured investment from Silicon Valley accelerator Alchemist and the University of Chicago. This funding coincides with ManiFast's final selection for Alchemist's 'Alchemist 2027' startup batch, with participation from the University of Chicago's Polsky Center for Entrepreneurship and Technology Commercialization. The specific investment amount was not disclosed.

ManiFast is a SaaS product targeting the pre-development phase of the software lifecycle. It uses AI to convert ideas or meeting notes into Product Requirement Documents (PRDs), feature specifications, user flows, and wireframes, which can then integrate with AI development tools like Claude Code and Cursor. The company states that over 2,000 enterprises and organizations currently use its product.

Under the plan, ManiFast will travel to San Francisco in January 2027 for a founder retreat and US entry program. Over approximately six months, the team will receive mentoring from local partners and sales coaches, culminating in a demo day presentation to US investors.

ソース:스타트업 생태계의 모든 것 \'플래텀(Platum)\'

リサーチ

Accenture and Amazon Web Services Propose a Three-Phase Sequence for Optimizing AI Token Costs, with Four Clients Reporting Reductions of 40% to 85%

重大
2026-11-20 02:48 GMT+8

A counterintuitive phenomenon often arises after enterprises scale their deployment of AI: while the price per token is falling, total token expenditure is rising. This occurs because cheaper tokens stimulate greater adoption, and agentic workflows split a single user interaction into multiple model invocations. A practice article jointly published by Amazon Web Services (AWS) and Accenture points out that in response to this situation, the correct optimization goal is not to reduce the number of tokens at any cost, but to match the appropriate level of intelligence to each business outcome—finding the corresponding point between cost, quality, and risk.

The article categorizes six optimization levers into three phases based on implementation complexity. Phase 1 involves pure configuration changes without altering infrastructure: prompt caching reuses repetitive system instructions and tool definitions, with design goals aimed at reducing cached input token costs by up to 90%; batch inference asynchronously processes non-urgent tasks in batches, costing 50% less than on-demand calls. Phase 2 involves architectural changes: intelligent prompt routing directs requests to different-sized models within the same model family, reportedly reducing costs by up to 30% without sacrificing accuracy; Retrieval-Augmented Generation (RAG) optimization reduces irrelevant context entering the model through hierarchical chunking, metadata filtering, and reranking. Phase 3 requires infrastructure and training investment: semantic caching stores semantically similar previously answered requests in vector storage; model distillation transfers knowledge from large models to small models, reportedly increasing speed by up to 500% and reducing costs by up to 75%.

The logic behind this sequencing is dependency. The article emphasizes that before applying any lever, cost attribution must be established—tagging by team, workload, and agent through Amazon Bedrock application inference profiles, then tracking consumption sources using Cost and Usage Reports (CUR) version 2.0 and model invocation logs. Without attribution, it is impossible to determine if optimization is effective or to distinguish whether a growing bill reflects success driven by business expansion or pure waste. To address this, Accenture proposed the Cost per Business Transaction framework, which aggregates all model invocations, retries, tool calls, and agent handoffs behind a single successful business output into one fully loaded cost figure, translating technical token consumption into metrics directly usable by business leaders.

Actual results from four clients confirm the sequence logic. A large enterprise previously routed all requests to frontier models with no caching, tiering, or batching; after Accenture applied model tiering, prompt caching, and batch processing simultaneously, monthly token costs dropped by 85%. A telecommunications operator with $12 million in annual token spend suffered major waste from agents fanning out requests without limits and sending all requests to frontier models regardless of complexity; after Accenture combined triage routing, semantic caching, and context compression, annual spend fell to $3.8 million, a 68% reduction. An insurance client and a mining client each applied only a single caching layer—the former using model-layer prefix caching, the latter using an application-layer semantic cache—and together they reduced token spend by approximately 40%, involving no model routing changes or training investment.

ソース:Amazon Web Services

リサーチ

Salesforce AI Research: The Challenge for Proactive Assistants Is Not Spotting Issues, But Deciding When to Stay Silent

クイックテイク
2026-11-20 00:11 GMT+8

In a blog post published on November 19, the Salesforce AI Research team introduced BLIP-Live, a research model designed for proactive video assistants. The team’s core assessment is that for AI providing real-time guidance in actual workflows, the true challenge is not identifying moments that need help, but determining whether its next message is worth interrupting the user’s ongoing actions.

This assessment stems from a specific scenario design. The article uses the example of a field service technician repairing a dishwasher: the technician’s hands are full, the next appointment is approaching, and a skipped inspection step could mean a return visit. If an AI assistant can issue a brief reminder at the right moment, the value is clear; however, if it repeats instructions for every routine action, it consumes the technician’s attention and may even cause frontline teams to reject the tool.

BLIP-Live’s training method directly addresses this balance. The team constructed large-scale training data that includes not only critical moments requiring a response, but also explicit periods of silence—intervals where the assistant should continue observing without speaking. The training objectives also include judgment after a response: if relevant states have not changed, the assistant needs to learn that the next message may be unnecessary. The team also explored the role of Agent Memory in reducing repetitive speech.

Evaluation design is another notable aspect of this article. The team points out that if evaluation metrics only reward catching events, they will favor an assistant that speaks constantly, creating a disconnect between good scores and poor user experience. Their evaluation measures both timely coverage and the proportion of responses credited for addressing an annotated opportunity. Repeated speech and messages outside the time window count toward the total number of responses, so unnecessary speech carries a clear cost. Meanwhile, an assistant that never speaks receives no credit either.

In self-tests across 326 first-person perspective videos, the team reported that BLIP-Live reduced unnecessary responses from state-of-the-art models by over 50%, and experiments showed human subjects preferred BLIP-Live’s responses more. These are vendor-reported results, with no independent replication and no public disclosure of specific annotation protocol details explaining how unnecessary responses were defined.

For readers, the practical incremental value of this article lies in the concretization of a design principle: the silence of proactive AI is not simple threshold filtering, but a capability that requires specialized training data and specialized evaluation metrics to shape. If you are designing any AI system that needs to judge when to intervene, treating silence periods as part of the training objectives and imposing costs on unnecessary speech in evaluations is a framework that can be directly borrowed.

The team stated that the next step is to validate this balance in real-world workflows, while measuring task outcomes and the experience of those receiving help.

ソース:Salesforce

リサーチ

Google Demonstrates Multi-Turn Tool-Calling Reinforcement Learning Training Dynamics: Interface Compliance Precedes Task Capability Convergence

クイックテイク
2026-11-19

The Google Developers Blog outlines a complete workflow for training multi-turn tool-calling LLM agents on Cloud TPU v6e-8 hardware. The experimental subject is the Gemma 4 E2B model (in thinking mode), operating within the Plane Strike board game environment—where players identify a hidden 8-cell plane on a 6×6 grid through successive strikes, with a maximum of 23 strikes per game. Training utilizes Tunix's agentic GRPO, collecting 64 multi-turn trajectories per step (8 boards × 8 games each).

What stands out most in this tutorial is not the final metrics, but the phased structure exposed by the training dynamics. The authors report that tool-call compliance (game/train/tool_call_rate) quickly approaches 100% early in training, while invalid actions drop to near zero and repeated strikes fall from approximately 0.55 to 0.15. Meanwhile, the actual win rate—the proportion of times the agent hits all 8 cells within 23 steps—gradually climbs from an initial training rollout average of roughly 0.20 (9.4% on held-out evaluation sets) to approximately 0.95, with average game length shortening from about 22 strikes down to 16. In other words, the model first learns to "operate the tool interface correctly," then learns to "use that interface to solve spatial reasoning problems."

This observation has practical implications for understanding agentic RL training: if tool-call compliance and task capability are two phases of separate convergence, monitoring only the final win rate during training may mask unresolved issues at the interface level. The authors also note that currently, all strikes share a single trajectory-level reward; finer-grained per-step credit assignment could yield further improvements—suggesting that the observed phased structure itself may be influenced by reward granularity.

It is important to emphasize that this is a self-reported demonstration by Google on a single game task, without independent replication. The rules of Plane Strike are simple, the action space is limited (36 cells), and the tool-calling pattern is singular (only one strike function), which differs significantly from complex, multi-tool, long-context production environments. Whether this phased structure in training dynamics reappears across other tasks remains to be verified.

ソース:developers.googleblog.com

リサーチ

Cisco Engineer Wins AWS AI League: Nine Single-Variable Experiments Reveal That Every Agent Handoff Consumes Priced Output Tokens

重大
2026-11-20 02:26 GMT+8

The 2026 AWS AI League hosted an agentic AI tournament at Cisco Transformation Fest 4.0, drawing 1,200 engineers. Champion Nishanth Aluhonnu’s design choices overturned a seemingly reasonable architectural convention: he defeated the competition’s reference architecture’s supervisor-plus-sub-agent pattern using a single agent.

Competition rules required agents to move across a grid map, solve various challenges (handling inappropriate content, running code, retrieving web information, extracting structured data), and reach a treasure chest. Scoring consisted of challenge points, treasure bonuses, remaining lives, and a token bonus calculated as 1,000 minus total output tokens divided by the number of visited challenges. This meant output tokens were a direct negative factor.

Aluhonnu built nine variants of the agent, changing only one factor at a time to isolate which tokens truly affected the score. The results were clear: tool return results and system prompts did not count against the score, but everything the agent wrote itself—including tool call arguments—was counted. Each tool call also carried approximately 52 fixed-overhead tokens.

This finding directly dictated architectural choices. The competition’s reference architecture showcased a supervisor agent coordinating three sub-agents (navigation, computation, web retrieval) routed to tools via AgentCore Gateway. Aluhonnu believed this pattern was reasonable for most production systems, but in this competition, every handoff between agents was a message the model had to write, and every written message consumed priced tokens. He switched to a single agent, placing a short dispatch table at the start of the prompt to map each challenge type to its corresponding method, completing routing within a single turn.

The second design decision was moving deterministic logic into tools. Route planning was not left to the model but ran in an AWS Lambda function, treating the board as an optimization problem: each tile had an expected value equal to the points it paid minus potential life costs minus the tokens the agent would spend solving it. The planner respected constraints (such as collecting keys before opening doors) and optimized routes within a time budget.

This principle received dramatic validation in the second round of the finals. A seven-by-seven board was almost entirely covered with coins worth 25 points each, while a scheduling challenge tile was worth 1,500 points. The habitual reaction was to collect coins, but the planner’s value table deemed them noise. The agent went straight for the high-value challenge, scoring 7,537 in that round, nearly 1,600 points higher than the next finalist’s 5,953. The lead established in this round ultimately secured the overall championship.

During the qualifier stage, the top four agents all solved the map. The first-place agent output 1,220 tokens, the second-place agent output 1,430, and the final margin was only 16 points out of a total score near 16,000. When all agents can complete the task, efficiency determines the winner. The article thus distilled a migration principle: let models handle language and judgment, and move search, mathematics, and rules into tools because they are faster, cheaper, and more repeatable.

ソース:Amazon Web Services

リサーチ

Unsloth Desktop Adds Local NPU Inference Support for AMD Ryzen AI Chips, Including Linux

クイックテイク
2026-11-19 22:25 GMT+8

U.S. semiconductor company AMD and AI model optimization developer Unsloth announced on November 19, 2026, that Unsloth Desktop now supports using Neural Processing Units (NPUs) in AMD Ryzen AI processors for local inference. This feature applies to AMD Ryzen AI laptops and desktops equipped with XDNA 2 and later architectures.

According to Phoronix, this integration leverages AMD's open-source Lemonade and FastFlowLM codebases for model serving and inference. More than 100 compatible agentic AI applications and workflows have been reported to work with Unsloth running on Ryzen AI NPUs.

Notably, AMD confirmed to media that this NPU support also works fine with Linux builds and is not limited to Microsoft Windows. This provides AI developers and enthusiasts using Linux with a new pathway to run large models more efficiently on consumer hardware locally.

ソース:phoronix.com

リサーチ

Ben Carlson: Stocks Outperform Bonds in 85% of Historical Decade Windows, but High Probability Does Not Equal Certainty

重大
2026-11-19 23:48 GMT+8

A reader asked Ben Carlson a question on stage at the Bogleheads conference: Which will have better returns over the next decade, stocks or bonds? The subtext of the question was that the U.S. stock market has risen for almost 18 years; surely mean reversion is due, right?

Carlson, a portfolio manager at Ritholtz Wealth Management, did not answer directly with a yes or no. Instead, he pulled up rolling ten-year return data for the S&P 500 and five-year U.S. Treasuries dating back to 1926. The results showed that stocks outperformed bonds in most decade windows, but not 100% of the time.

He approached the quantification of this issue from another angle: using the current government bond yield of approximately 5% as a benchmark line, stocks have historically exceeded this level in 85% of all ten-year time frames. If you hold a broad-based stock index fund for ten years, there is an 85% probability that your annualized return will exceed the coupon income of Treasury bonds during the same period.

But frequency is only half the story. Carlson further compared the magnitude of difference in performance when each asset class wins: when stocks outperform bonds, the margin of victory is far greater than when bonds outperform stocks. This means that even if you occasionally pick the wrong side, your losses are relatively limited; whereas when you pick the right side, the gains are significantly higher.

Based on this asymmetry, Carlson’s conclusion is that if he had to place a bet in Las Vegas on who would win over the next decade, he would pick stocks every time because the odds are in his favor. However, he simultaneously emphasized that high probability does not equal certainty. That 15% of failure periods were not pleasant for investors. He cited a metaphor from Bill Bernstein at the conference: holding the market for the long term makes you the house because the odds are in your favor; but even the house loses sometimes.

Carlson did not frame this issue as an either/or choice. He pointed out clearly that the good news is you do not need to bet on one or the other. Holding bonds is not suitable for everyone, but fixed income can provide a cushion when you lose a hand or two. Although the current 5% yield level does not guarantee that bonds will definitely earn 5%, the starting yield is a good approximation of future returns—meaning that even if interest rates continue to rise, once the dust settles, expected returns could actually be higher. His advice is essentially: take on stock risk when probabilities are favorable, while using bonds to provide a margin of safety when truly needed.

ソース:A Wealth of Common Sense

リサーチ

AWS Launches Cross-Region Inference Routing for SageMaker AI to Mitigate Capacity Shortfalls

重大
2026-11-20 02:20 GMT+8

Amazon Web Services (AWS) announced the general availability of cross-Region inference routing in Amazon SageMaker AI on November 19, 2026. This new resource enables applications to use a single invocation path across endpoints and inference components deployed in one or more AWS Regions and accounts, addressing service interruptions caused by GPU capacity shortages in specific regions.

The core mechanism relies on an 'inference router.' Users register multiple destinations serving the same model, and the application invokes the router instead of a specific endpoint. The router continuously evaluates each destination's 'health score' based on recent request outcomes, such as server errors (5xx), throttling, and timeouts. Traffic remains local when the primary endpoint is healthy; upon detecting unreliability, the router automatically shifts new requests to registered remote destinations and gradually returns traffic after recovery. Notably, model latency alone does not trigger failover; traffic only moves when requests begin returning errors or timing out.

For enterprise architects, this update eliminates the need to build custom cross-region failover code. Previously, teams had to manually detect endpoint stress, decide retry logic, and manage cross-account access permissions. These operations are now handled by the SageMaker AI platform layer. Additionally, AWS explicitly states that there is no extra charge for this feature; users pay standard rates only for the endpoints and inference components that actually serve their requests.

However, the feature is not a silver bullet. The router does not create new compute capacity or replicate model artifacts. It depends on users having already deployed endpoints with sufficient headroom in different regions. If all remote destinations also reach their capacity limits, requests will still be throttled. Therefore, enterprises must still plan for multi-region capacity redundancy, though operational complexity is significantly reduced.

ソース:Amazon Web Services

リサーチ

MaxText Streaming DiLoCo: Fragmenting Models for Step-by-Step Synchronization to Eliminate Network Wait in Cross-Datacenter LLM Training

重大
2026-11-19

Traditional large-scale model training relies on a tightly coupled accelerator cluster where all devices synchronize gradients at every step via high-bandwidth interconnects. When physical and cost constraints make it difficult to continue scaling a single cluster, deploying compute slices across geographically distributed data centers is the natural next step. However, standard data parallelism degrades severely over wide-area networks (WAN): every step requires transmitting the gradients for the entire model, and because WAN bandwidth is far lower than internal data center interconnects, transmission cannot be fully overlapped with computation, causing training to stall while waiting.

DiLoCo (Distributed Low-Communication), proposed by teams from Google DeepMind and Google Research, focuses on reducing synchronization frequency. It divides training into M independent learner groups, each running H steps of local optimization (typically using AdamW) on its own data without communicating between groups. After every H steps, each group calculates weight deltas, averages them via an all-reduce operation, and updates global weights using an outer optimizer (SGD with Nesterov momentum). This reduces communication volume by a factor of 1/H, with papers reporting convergence quality comparable to standard data parallelism.

However, Vanilla DiLoCo has a lingering issue: although total communication is reduced by a factor of H, each synchronization event still transmits the weight deltas for the entire model. On narrowband links, this creates large traffic spikes once every H steps. Taking Qwen3-32B (approximately 32 billion parameters, bfloat16) as an example, the full model is about 65.5 GB. On a 10 Gbps link, transferring 65.5 GB takes over 50 seconds, whereas a normal calculation step takes about 6 seconds—resulting in a 50-second pause every 65 steps.

The solution offered by Streaming DiLoCo is to cut model parameters into P disjoint fragments, synchronizing exactly one fragment per step within the H-step window. When P = H, each step synchronizes 1/P of the model, presenting the network with a nearly uniform stream of small traffic rather than periodic large spikes. MaxText's fragmentation method sets P = N_layers + 1: each Transformer decoder layer constitutes one fragment, while all non-scanned parameters (token embeddings, final layer normalization, and output head) form another fragment.

There is an easily overlooked engineering detail here: the size of the embedding fragment depends on the vocabulary dimension and can be much larger than that of a single-layer fragment. If unaddressed, the step synchronizing this fragment would still generate a traffic spike. MaxText addresses this by evenly splitting and merging large matrices from the embedding fragment into other layer fragments, ensuring all fragments are roughly equal in size and truly achieving a uniform flow.

The synchronization order also matters. MaxText does not synchronize adjacent layers sequentially but instead interleaves fragment selection (e.g., synchronizing fragment 0, then layer k, then layer 2k...). This ensures that every round of synchronization touches different depths of the network, preventing any section of the model from remaining stale for too long while other sections update.

ソース:developers.googleblog.com

リサーチ
次へ →
ログイン / 新規登録
読書リサーチレーダー投資フレームワーク

The article recommends the following operational order: first connect consumption to owners and business outcomes, then apply one lever to one workload at a time, measure cost, quality, and business value, and proceed further based on what works. For regulated industries (financial services, insurance, healthcare), quality test results and rollback procedures should be included in standard change management records.

It is important to note that these conclusions are bound to specific scoring rules. The competition treated output tokens directly as a negative factor, and tool call arguments counted toward outputs. In production environments, if token costs represent a different proportion of total costs, or if task complexity requires the modular maintenance advantages of sub-agents, multi-agent architectures may still prevail. The article summarized this by stating: the lesson is not that sub-agents are bad, but that you should choose your architecture based on coordination costs.

Roofline modeling allows for precise calculation of bandwidth requirements. Standard data parallelism transfers 2|Θ|b bytes per step (where |Θ| is the parameter count and b is bytes per parameter), requiring completion within the backward propagation time t_bwd, resulting in a peak bandwidth requirement of approximately 2|Θ|b / t_bwd. For Qwen3-32B under bfloat16, this is about 85 Gbps. Streaming DiLoCo transfers only 1/P of the model per step, lowering the bandwidth requirement to approximately 85/P Gbps. When P = 65, this drops to about 1.3 Gbps—more than sufficient for a 10 Gbps link.

The Google team simulated three network conditions on two TPU v5p-32 slices using Linux tc traffic shaping. At 30 Gbps, data parallelism degraded to 19.41 seconds/step, while Streaming DiLoCo maintained 5.76 seconds/step (a 3.37x speedup). At 10 Gbps, data parallelism degraded to 55.11 seconds/step, whereas Streaming DiLoCo remained at 5.76 seconds/step (a 9.57x speedup). Vanilla DiLoCo at 10 Gbps ran at 7.55 seconds/step—much better than data parallelism but still suffering from periodic pauses every 65 steps.

Regarding convergence quality, the team ran 4,000 steps of pre-training on two MoE architectures (Qwen3-30B-A3B and Gemma4-26B-A4B). After a brief adjustment period in the first ~800 steps, the loss curve for Streaming DiLoCo smoothly tracked and slightly outperformed the synchronous data parallelism baseline. Stable convergence relies on two design choices: the outer loop uses Nesterov momentum (β=0.9, learning rate 0.3–0.7) rather than simple averaging; and the inner loop's AdamW first- and second-moment buffers remain unchanged after fragment synchronization, ensuring learners do not lose optimizer state.

It must be emphasized that all the above performance figures come from self-tests on Google's proprietary TPU clusters, using Linux tc to simulate network constraints rather than real cross-domain deployments. The 9.57x speedup appears in the extremely constrained scenario of 10 Gbps. Actual production network conditions, topologies, and traffic patterns may differ. However, its core mechanism—breaking large synchronizations into uniform small flows to match narrowband links—is a transferable distributed systems pattern that does not depend on a single high-bandwidth cluster.