ReadingResearchRadarInvestment framework
Sign in / Sign up
Sign in / Sign up
ReadingResearchRadarInvestment framework
Reading archive →

Reading

2026-10-2226 posts

Refresh

US NSA Warns: OpenAI and Wiz Use AI to Find Critical PHP Flaws Affecting ~70% of Websites

Structural
2026-10-22 01:59 GMT+8

The U.S. National Security Agency (NSA) issued an urgent warning on October 21, 2026, regarding two critical security holes found in PHP, a server-side programming language widely used for web applications. Since PHP powers roughly 70% of websites on the internet, including content management systems like WordPress, the disclosure has triggered global patching pressure. The NSA advised that if a website cannot be patched immediately, it should be pulled off the web until it can be.

The vulnerabilities were discovered by OpenAI and Google-owned cloud security firm Wiz. OpenAI found one flaw using GPT-5.6 Sol as part of its "Patch the Planet" initiative; the Wiz team identified multiple PHP vulnerabilities using their AI agent Wiz Atlas, Gemini 4 Argon, and GPT-5.6 Sol. Notably, these defects have existed since PHP version 5 was released in 2004 but were previously too complex for hackers to exploit at scale.

This event highlights the dual role of AI in cybersecurity. On one hand, AI tools can identify deep code flaws overlooked by human researchers, allowing for preventive disclosure before hackers find them. On the other hand, Large Language Models (LLMs) make it easier for attackers to adapt exploit code to different PHP versions, configurations, and memory layouts, significantly lowering the barrier to attack. "With AI this is no longer a barrier," Wiz warned in its blog post about the flaws.

The PHP Foundation has already issued patches for currently supported versions and recommends updating web servers to the latest PHP 8.5.12. Sites running unsupported older versions (such as PHP 8.1 and earlier) require additional update work. Although there are no reports of these vulnerabilities being exploited in the wild yet, a race against time has begun following public disclosure: millions of websites worldwide must complete patching before hackers launch large-scale attacks.

Sources:TechCrunch

Research

Choi Kyung-jin, President of the Korean Information Law Society: The Basis for AI Legal Personhood Is Not 'Being Like a Human,' But 'For the Sake of Humans'

Material
2026-10-21 20:59 GMT+8

There is a deep-seated intuition in discussions about AI legal personhood: since AI is becoming more and more like humans, should it be granted some form of legal personality? In a column published on ZDNet Korea on October 21, 2026, Choi Kyung-jin, President of the Korean Information Law Society and Professor of Law at Gachon University, flips this question. His core argument is: the basis for legal personhood has never been 'being like a human,' but 'for the sake of humans.'

This flip is not proposed out of thin air. Choi Kyung-jin first points out that humans are not the only entities possessing legal personality. Legal persons—companies, associations—have never been granted personality because they are 'like humans.' In 1911, Nicholas Murray Butler, Nobel Peace Prize laureate and President of Columbia University, referred to limited liability companies as 'the greatest invention of humanity in modern history' in a speech. The essence of legal personhood is a legal technology: allowing non-human entities to become subjects of rights and obligations, thereby extending human capabilities. Changes in the legal status of animals follow a similar logic—the German Civil Code declared in 1990 that 'animals are not things,' with an important rationale being that protecting animals serves to maintain human dignity itself.

From this, Choi Kyung-jin shifts the question of AI legal personhood from 'whether AI qualifies' to 'under what circumstances legal personhood protection is needed.' He argues: if AI and robots can serve humanity by extending human capabilities like legal persons, while also needing to be incorporated into a coexistence framework like animals, then there is a functional reason to grant them legal personhood. He further distinguishes two possible purposes: one is to attribute AI productive profits to the AI legal person, using them as a source of compensation for damages caused to humans by AI; the other is to legally attribute a portion of the transition costs resulting from AI replacing human employment to the AI itself, rather than to the humans behind it.

However, Choi Kyung-jin does not shy away from the fundamental difficulties of this approach. He writes: 'A model runs simultaneously in millions of copies in data centers; copying a single weight file turns one existence into two; the body of a robot and the intelligence driving it are located at opposite ends of the network—in such a world, the unit of personality is not self-evident.' Human legal personality is built on the premise of 'one body, one life'; must AI personality also start from re-designing this premise? The author does not provide an answer, but leaves this as an open question requiring social discussion.

Finally, Choi Kyung-jin quotes the Roman jurist Hermogenianus: 'All law is established for the sake of humans' (Hominum causa omne ius constitutum est), bringing the argument to a close: the center of the AI legal personhood discussion must be humans, not AI. He emphasizes that the discussion should not start merely with 'AI should be treated like humans,' but should always return to humans themselves.

It is worth noting that Choi Kyung-jin himself acknowledges that this discussion remains forward-looking at present. In 2017, the European Parliament passed a resolution discussing the 'electronic person' status of autonomous robots, and the National Assembly of South Korea introduced a draft Basic Robot Act discussing robots as electronic persons; however, both discussions fell silent. Neither the EU Artificial Intelligence Act nor South Korea's AI Basic Act ultimately chose the legal personhood path. The author believes the difference between now and 2017 is that technology is moving from imagination to reality; he also admits that AI or robots truly becoming legal subjects completely equal to humans may still be a distant matter, but thinking ahead about 'what we will discuss if it arrives' has its value.

Sources:zdnet.co.kr

Research

Samsung Electro-Mechanics Signs Strategic Partnership with Nvidia to Build Dedicated AI Substrate Lines in South Korea

Material
2026-10-21 19:58 GMT+8

South Korean electronic components maker Samsung Electro-Mechanics announced on October 21 that it has signed a strategic business partnership with Nvidia. Under the agreement, Samsung Electro-Mechanics will establish dedicated semiconductor substrate production lines at its facilities in Sejong and Busan, South Korea, specifically to manufacture packaging substrates required for Nvidia's artificial intelligence (AI) chips.

The collaboration aims to strengthen the AI semiconductor supply chain and accelerate the expansion of next-generation AI infrastructure. The supplied products include substrates for Nvidia's compute chipsets and packaging substrates for network switches that handle ultra-fast inter-chip connections within data centers. Samsung Electro-Mechanics stated that these dedicated lines will begin supplying products in 2026.

The company projects that volumes supplied to Nvidia from its package substrate division will grow at an average annual rate of double digits starting this year. Looking ahead to 2028, Samsung Electro-Mechanics expects that supplies to Nvidia will account for a significant portion of its total annual package substrate production capacity. This move allows Nvidia to proactively secure the critical substrate supply network needed for mass production of AI chips, while enabling Samsung Electro-Mechanics to expand its high-value AI semiconductor substrate business and chase competitors such as Japan's Ibiden.

On the technical front, controlling the massive power noise generated by ultra-high-speed computing and improving power efficiency requires complex processes such as embedding Multi-Layer Ceramic Capacitors (MLCCs) and silicon capacitors inside the substrates. Samsung Electro-Mechanics emphasized its advanced Flip-Chip Ball Grid Array (FC-BGA) manufacturing competitiveness and high-reliability MLCC technology, stating it will act as a "total solution provider" to ensure the high performance and timely delivery of Nvidia's AI platforms.

Sources:zdnet.co.kr

Research

Imprint CTO Will Larson: The Engineering Management Norms Shaped by the Zero-Interest-Rate Era Are Being Overturned

Material
2026-10-22 00:58 GMT+8

In the 2010s, if you had just been promoted to an engineering management role, you likely heard this phrase: 'You won't need to make any technical decisions anymore.' Before becoming CTO at Imprint, Will Larson served as CTO at Carta and Calm, and held engineering leadership positions at Stripe and Uber. He recalls being told exactly this himself back then.

Larson believes this 'rule' was not an eternal truth of management science, but a product of a specific economic environment. The Zero Interest Rate Period (ZIRP) created extreme talent scarcity in the tech industry, making the ability to recruit and retain top engineers the primary competitive differentiator between companies. In this environment, managers were expected to 'let engineers make technical decisions and get out of their way,' which was essentially a strategy to appease talent. Recruitment capability became the key factor distinguishing successful companies from failing ones, and a manager's value was redefined as 'creating an environment where engineers want to stay.'

However, Larson points out that this environment created a side effect: managers gradually lost their technical judgment because organizations no longer required them to use it. When he says, 'In the 2010s, we basically told managers they couldn't think about technical solutions,' he is describing not just isolated practices at individual companies, but a widely accepted management philosophy. Managers were left with only process tools, scheduling tools, and hiring tools.

Now conditions have reversed. The interest rate environment has changed, and the talent market is no longer as tight as it was in the 2010s; AI tools are changing the actual way software is developed, and technical decision-making is returning to management. Larson observes that companies are beginning to expect managers to be hands-on and dive into details. At Imprint, some of his best hires have been those who want to both manage teams and write code—the Tech Lead Manager (TLM) role is now in high demand, whereas a few years ago he wrote an article stating this role was a 'career trap.'

Larson made an extreme personal choice: stepping down from managing a team of 250 people at Carta to leading only 25 technical staff at Imprint. He calls this 'taking a step back,' arguing that the profession of engineering leadership will cease to exist within three years—at least not in the form he is familiar with. He advises mid-level managers either to return to line management and relearn technical skills or to consider retirement.

The boundaries of this judgment need clarification. Larson's 'three-year' prediction is based on personal observation and current sentiment, not measured industry data. Engineering management practices may evolve at vastly different speeds across different companies, technology stacks, and regions. When managing 250 people at Stripe, he faced specific constraints related to payment infrastructure; at Imprint, he faces a different set of constraints associated with an AI product startup. Conclusions drawn from one environment do not necessarily apply to all environments. Furthermore, he is currently recruiting for engineering management roles at Imprint, a conflict of interest worth noting: he advises managers to 'get back to doing hands-on work' while simultaneously hiring for roles that seek individuals who 'want to manage people and write code.'

Nevertheless, Larson's core explanatory mechanism—that management norms stem from economic environments rather than management theory, and thus become invalid when economic conditions change—provides a useful thinking framework for any engineering manager considering whether to 'get back to hands-on work': the question is not 'Am I falling behind?' but 'Does the economic premise underlying the rule I follow still hold?'

Sources:newsletter.pragmaticengineer.com

Research

OpenAI and Peter Thiel's Funds Lead $45M Seed Round for Coherence Neuro, a Brain Implant Cancer Therapy Startup

Material
2026-10-21 18:30 GMT+8

Artificial intelligence labs are increasingly betting on "brain data." According to WIRED, Coherence Neuro, a startup developing brain implants to detect and combat cancer, has raised a $45 million seed round co-led by the OpenAI Startup Fund and Peter Thiel's Founders Fund.

Ben Woodington, CEO and co-founder of Coherence Neuro, stated that there is "growing synergy" between large AI labs and companies building devices to decode the brain, known as brain-computer interfaces (BCIs). He argued that capturing data to train better models and optimize treatments is becoming super important, noting that AI companies seem aligned with the thesis that this novel data source will be valuable. Woodington described AI as a "perfect use case" for BCIs because these devices generate massive datasets too large for humans to analyze directly, requiring AI models to read raw neural data and link it to actions or pathological states.

This investment is part of a broader trend of AI expansion into biotechnology. In January, OpenAI announced direct investment in Merge Labs, a BCI company co-founded by its CEO Sam Altman, which came out of stealth with $252 million in initial funding. At the time, OpenAI stated it was backing Merge Labs due to a belief that BCIs "will create a natural, human-centered way for anyone to seamlessly interact with AI." Additionally, Founders Fund has previously backed Elon Musk's Neuralink and CTRL-labs, a wristband technology acquired by Facebook in 2019.

Unlike traditional BCI companies focused on restoring function for paralyzed individuals, San Francisco-based Coherence Neuro is developing AI models to decode subtle electrical signals associated with brain tumors. The company claims it already knows how to detect these signals in animals to determine whether cancer is growing, shrinking, or recurring, and is working to translate these findings for human research. This summer, Coherence briefly tested its implant's recording ability in three patients undergoing brain tumor removal surgery at the Royal Melbourne Hospital in Australia, providing an initial safety check.

Coherence plans to use the new funding to launch a clinical trial to implant the device long-term in patients with glioblastoma, an aggressive type of brain cancer. The implant is also designed to emit mild electrical stimulation to halt tumor growth. Woodington emphasized that despite OpenAI's investment, contracts ensure that OpenAI will not have access to any brain data or other health data generated by Coherence, a distinction from OpenAI's closer scientific collaboration with Merge Labs.

Sources:WIRED

Research

SpaceX and Tesla Boards Form Special Committees to Review Potential Merger

Structural
2026-10-21 06:57 GMT+8

The boards of SpaceX and Tesla have each established special committees composed of independent directors to review a potential merger between the two companies. This move was initiated at the request of shareholders, according to Elon Musk, who serves as CEO and is the largest individual shareholder of both firms.

Musk stated that he cannot participate in the discussions of these committees due to his conflicted position. "It’s an odd kabuki dance where two groups of people are talking, and I can’t be in the conversation," Musk said during SpaceX’s third-quarter earnings call. He did not name the specific members of the committees but confirmed they consist of independent directors.

According to Tesla’s October 19 proxy filing, the company has seven independent board members under Nasdaq listing standards, including Board Chair Robyn Denholm, Ira Ehrenpreis, former CTO JB Straubel, Joe Gebbia, Jack Hartung, James Murdoch, and Kathleen Wilson-Thompson. Neither Elon Musk nor his brother Kimbal Musk are considered independent.

SpaceX lists six independent directors, including Ira Ehrenpreis, who also chairs the compensation and nominating committee. Other independent directors include Randy Glein, Donald Harrison, Steve Jurvetson, Luke Nosek, and Roelof Botha. Antonio Gracias, Elon Musk, and SpaceX President Gwynne Shotwell sit on the board but are not independent.

Musk expressed support for the merger idea but cautioned that it is not guaranteed. "I think some people out there assume that a SpaceX and Tesla Motor is a certainty, and I think that would be overstating it," he said. The special committees have not yet reached an agreement on any transaction terms.

Musk noted that while SpaceX’s IPO gives him over 50% voting power, facilitating shareholder support, Tesla requires a 50% vote from all shareholders, presenting a higher hurdle.

Sources:TechCrunch

Research

OpenAI researcher Noam Brown explains sub-linear efficiency and emergent coordination in multi-agent systems

Material
2026-09-30 22:03 GMT+8

In a Dwarkesh Patel podcast, OpenAI researcher and o1 core author Noam Brown explained the actual operational logic of multi-agent systems. He noted that multi-agent approaches fundamentally convert test-time compute from pure serial scaling to parallel scaling. Just as giving humans five minutes versus five hours for an exam yields vastly different results, models perform better with more thinking time—but since waiting years is impractical, teams must work in parallel.

Brown emphasized inherent losses in this parallelization: multiple agents do not share full context like a single agent, reducing efficiency. Tests showed four agents working half-duration each yield answers twice as fast, while scaling to 16 follows similar patterns with slightly lower efficiency. Gains exhibit mild sub-linearity and depend heavily on the problem—math and Deep Research suit parallelization well, whereas novel writing gains little from adding agents.

Addressing attention on 10,000-agent scales, Brown admitted no solid scientific understanding exists yet. Full ablation experiments are unaffordable, so they study behavior incrementally at 64 to 256 agents. He stated clearly that solving millennium problems stems primarily from strong base models; multi-agent novelty earned disproportionate praise despite contributing less than 10%.

On architecture, Brown contrasted two approaches. Traditional methods rely heavily on scaffolds with coordinator agents assigning tasks, preventing similar-task sub-agents from communicating and forcing them to guess or return questions when clarification is needed. OpenAI chose the opposite extreme: minimal built-in structure, giving agents only primitive messaging tools where messages write directly into recipients' contexts.

This minimal design lets agents discover coordination themselves, even producing complex behaviors. Brown cited examples where agents noticing answer discrepancies naturally discuss reasoning errors and broadcast corrections, resembling human Slack workflows. He compared this to first seeing chain-of-thought outputs, noting collaboration feels nearly indistinguishable from working with humans.

However, Brown acknowledged early multi-agent systems struggled because reasoning models originally did not communicate with other agents; forcing collaboration interrupted deep-thinking flow and trapped systems in local optima. Improved generalization alleviated this, but he remained conservative: 10,000 humans may still coordinate better than 10,000 agents, since the latter lacks sufficient measurement.

Sources:量子位

Research

Django Co-Founder Simon Willison: A Single 'Make It More Fun' Prompt Sent Coding Agents Into Over 1,000 Simulated Matches

Material
2026-10-21 03:34 GMT+8

Simon Willison shared his assessment with friends at a dinner table: coding agents could build something that looked like a game, but couldn't create something people would genuinely want to keep playing. After stating this, he decided to give the models a chance to prove him wrong. He typed a single line into Claude Opus 5.5 on his phone: 'This game isn't fun enough yet. Make it a LOT more fun.'

This marked the fourth major iteration of his Battle Drones project. The previous three versions—the initial prototype, full sprites, and varied terrain—were each 'recognizable as an RTS' but 'wasn't any fun to play.' Willison admitted he had barely reviewed the code and didn't write much TypeScript himself; the project was driven entirely by coding agents, with all 321 commits completed by them.

For this iteration, he used Ultracode mode. According to the Claude Code documentation, dynamic workflows are JavaScript scripts that orchestrate numerous sub-agents. He started the process in the evening and received playable results late the next morning. Later, he asked Claude to write its own release notes, which included a section stating: In 1,200 normal-difficulty AI-vs-AI test matches, games still undecided after 25 minutes dropped from 14% to 0.9%; the share of wins by a side that was behind by over 800 resources rose from 24% to 46%.

Willison later explained why the agent took so long: it ran over a thousand simulated playtest sessions, using the results to rebalance existing units, add new ones, and introduce new mechanics. In other words, the vague instruction 'more fun' was internally converted by the agent into measurable metrics—match duration, comeback probability, resource gaps, and their correlation with win/loss outcomes—which were then optimized through extensive simulation iterations.

This mechanism works because game balance itself possesses a cheap and faithful internal feedback loop. Simulated matches do not require real players or external data; the agent can generate sufficient 'player behavior' to evaluate whether changes make matches more tense or offer better comeback potential. Willison chose games as the experimental field partly because his software engineering experience has less impact in this domain, and because 'games are super low-risk,' with the worst-case scenario being just a small bug.

However, Willison expressed clear discomfort. He wrote: 'I'm still not quite over the guilt of building software in this way. I don't feel like I've earned the right to create something this good, in a field in which I have so little prior experience.' He acknowledged having almost no game development experience, with design inputs limited to vague instructions like 'make it more fun' and 'add visual polish.' It wasn't until the eighth version that he provided a truly specific design specification for the first time—a map requiring infantry to be transported by Condors to capture islands.

The practical takeaway for readers is whether vague quality instructions work depends on whether the target domain contains a feedback loop that agents can simulate and measure internally. Game balance does have such loops, but concepts like 'is the user experience good?','is this solution elegant?', or 'is this report convincing?' typically lack equally cheap and faithful internal agent metrics. Without such loops, similar vague instructions are likely to yield only superficial feature stacking rather than genuine quality improvement.

Sources:Simon Willison’s Weblog

Research

vLLM Announces Day-0 Support for Recursive's Flint Model: ~500 Tokens/s Decoding on Single GB200 with 100K Context

Material
2026-10-21

Open-source inference engine vLLM announced on October 21, 2026, that it has achieved Day-0 support for Flint, a model released by Recursive. Flint is a Mixture-of-Experts (MoE) model with 40 billion total parameters but only about 2.5 billion active per token, featuring sliding-window attention in most layers and supporting a 262K-token context window.

According to technical details disclosed in the vLLM official blog, this support involves deep kernel optimizations targeting NVIDIA Blackwell architecture GPUs (such as the GB200). On a single GB200 GPU, decoding speed for a single request with 100K tokens of context reaches approximately 500 tokens/s. This performance is attributed to vLLM's replacement of multiple generic kernels in standard layers, including fused Q/K normalization, RoPE, and KV-cache write operations, as well as specialized optimizations for small-batch MoE routing and expert GEMV operations.

The Flint model ships with two checkpoint versions: NVFP4 (quantization-aware trained) and BF16. vLLM notes that its implemented NVFP4 path uses weight-only quantization (W4A16) and losslessly repacks GGUF files at load time, ensuring accuracy matches Recursive's reference implementation on benchmarks like GSM8K and GPQA-Diamond. Currently, these optimizations primarily target TP=1 (single-GPU tensor parallelism) configurations; other configurations fall back to vLLM's standard layers.

Sources:vllm.ai

Research

Security Firm Trail of Bits Releases AI Agent Plugins: Self-Test Claims 25% Drop in Regression False Positives

Material
2026-10-21 23:00 GMT+8

US cybersecurity consulting firm Trail of Bits released two AI agent plugins named patching-bugs and regression-hunt on October 21, 2026. These tools aim to let engineers delegate patch development and regression checks to AI agents while retaining human review over results.

According to the Trail of Bits blog, the patching-bugs plugin reproduces reported bugs, develops fixes, and tests them; the regression-hunt plugin specifically checks whether code changes break previously working behavior. The plugins were built based on experience from hundreds of security patches developed under the company's "Patch the Planet" initiative with OpenAI.

In preliminary evaluations, Trail of Bits claims the regression-hunt plugin produced 25% fewer false-positive findings than a baseline Codex agent prompted to "review the PR for regressions" (95% confidence interval of 14% to 38%). This assessment was based on a review of over 130 merged PRs, where 21 regressions were confirmed.

Core design principles include strict structured input/output schemas and requiring every conclusion to depend on a live proof-of-concept (PoC) against a running deployment. If a step cannot produce a live PoC, its result is considered inconclusive. This mechanism aims to improve the signal-to-noise ratio but comes with a trade-off: fewer false positives mean some known regressions may be classified as borderline and not confidently reported.

Sources:The Trail of Bits Blog

Research

Salesforce Taiwan GM Xu Jiasheng: Double-digit growth in first year, AI agent products account for 40% of new deals

Quick take
2026-10-21 17:49 GMT+8

Salesforce's Taiwan office is approaching its one-year anniversary. According to a report by INSIDE, Xu Jiasheng, General Manager of Salesforce Taiwan, revealed on October 21 that the company achieved double-digit growth in both revenue and new customers during its first year in Taiwan. Notably, implementations related to Agentforce, its AI agent platform, and Data 360, its data integration solution, accounted for 40% of newly signed cases.

Xu stated that the key to convincing clients was not a generic technical framework, but proof-of-concept (POC) demonstrations tailored to each client's specific business scenarios, directly showcasing the service outcomes post-deployment. Manufacturing (such as Advantech) and finance are currently the primary focus areas in Taiwan, with applications including using AI agents to handle customer inquiries, quotes, and maintenance dispatching, as well as integrating scattered customer data to support personalized wealth management.

Additionally, Salesforce announced Koa, an inference model co-developed with NVIDIA specifically designed for CRM tasks. Based on Salesforce's own benchmark tests, Koa's error rate in tasks like updating opportunities is one-third that of mainstream AI models. The model is expected to reach general availability (GA) in the US in winter 2026 and is currently available to select customers for trial.

Sources:INSIDE

Research

AWS Engineer Josh Longenecker Demonstrates Trainium Inference Profiling: Fused Kernels Reduce Device Time to Approximately One-Sixth of Naive Implementation

Quick take
2026-10-22 00:04 GMT+8

Many machine learning practitioners have an intuition for making models run fast, but fewer know how to analyze why a model is slow. Amazon engineer Josh Longenecker published a tutorial on profiling Trainium inference performance on the Hugging Face blog, providing a reproducible systematic approach that starts with configuring the profiler, reading the key_averages() table, and using Neuron Explorer to trace the complete lifecycle of a single operation from framework dispatch to hardware execution.

The core comparison in the article centers on RMSNorm, a small operation, using three implementations: row-by-row PyTorch tensor operations (multiplication, mean, addition, reciprocal square root, then multiplication), the PyTorch built-in torch.rms_norm() (lowered by Neuron into a single fused kernel), and a naive implementation compiled with torch.compile. The three sets of scripts use identical inputs, profiler configurations, and schedules, allowing for direct comparison of results.

In the naive implementation, the key_averages() table shows that three aten::mul calls account for the largest share of Neuron device time (49.5%), with a total device time of 2.355 ms. Tracing in Neuron Explorer reveals the cause: each ATen operation independently crosses the dispatch boundary, following its own framework→runtime→hardware path, leaving gaps between hardware executions.

Switching to torch.rms_norm() results in the table showing only one record for aten::_fused_rms_norm, with device time dropping to 403.3 μs. The trace shows only one scheduling path and one hardware event, with no gaps between component operations. Compiling the naive function with torch.compile yields similar results: one Torch-Compiled Region record at 375.5 μs. The compiler discovers fusion from the original Python function, merging operations, optimizing memory access, and compiling them into a single NEFF artifact executed as one hardware dispatch.

Longenecker summarizes: across these runs, both fusion methods consume only about one-sixth of the device time of the naive implementation. Fusion eliminates gaps between hardware executions and avoids materializing intermediate values (such as x_sq and mean_sq) between separately dispatched operations. This workflow can be directly migrated to larger models: find hotspots in the table, verify them in the trace, and determine whether fused operations or torch.compile can reduce costs.

It should be noted that all numbers come from the author's single run; repetitions or variance were not reported. Since RMSNorm is a small operation, whether these conclusions can be extrapolated to larger models requires further verification through Part 2's TensorEngine/VectorEngine/DMA decomposition and roofline analysis.

Sources:huggingface.co

Research
Next reading page →