ReadingResearchRadarInvestment framework
Sign in / Sign up
← Back to reading
NewsMaterial4 min readDisconfirmAI Editorial

Chip Architecture Allocates Memory by Workload: Chipstrat Analyst Austin Lyons Argues Micron's 'More Memory is Better' Answer Misses the Point

After attending a Micron Technology product webinar, Chipstrat analyst Austin Lyons pointed out that Micron executives' response to the question of "despeccing"—claiming that "more memory equals better system performance"—avoided the real issue: how total memory demand and structure will change when training, prefill, and decoding use different chips and different types of memory. Citing examples such as Google's TPU 8i/8t, Nvidia's Rubin CPX, and Positron's Asimov, he noted that customers are already precisely configuring memory according to workloads. Since HBM consumes three times the wafers per bit compared to DDR5 and currently has lower margins, migrating bits from HBM to LPDDR/GDDR could improve rather than harm Micron's profits.

2026-10-27 23:30 GMT+8

Micron held its Technology and Product Webinar last week. Chipstrat analyst Austin Lyons watched it and recorded his analysis, with the core judgment being: Micron gave an answer that was correct but missed the point regarding the memory demand question most concerning investors.

The starting point of the question came from analyst Vivek Arya. He asked a crucial question for memory manufacturers: today, memory accounts for more than half of an accelerator's bill of materials. Customers are discussing "despeccing," i.e., using less or cheaper memory. Could inference accelerators eventually be equipped with less DRAM or different types of DRAM? If so, Micron's bit sales would decline.

Micron's Scott DeBoer answered that system performance improves with increased memory capacity and bandwidth. Customers will choose the minimum specification that meets their product and business needs. Micron's job is to increase supply and drive bandwidth higher through products like vertical HBM. He concluded with: "Inherently, more memory does equal better system performance."

Lyons believes this answers the wrong question. DeBoer is right—more memory does help performance—but Arya didn't ask "is more memory better?"; he asked "what does it mean for memory suppliers when customers configure memory precisely according to workload?"

Sign in / Sign up
ReadingResearchRadarInvestment framework

Lyons uses Google's chip design to illustrate that customers are already doing this. Publicly available specifications for Google Cloud's TPUs show: the inference chip TPU 8i has less compute but more memory; the training chip TPU 8t has more compute but less memory bandwidth because training is primarily compute-intensive. This is not hypothetical; it is differentiated products already in the market.

He further breaks down inference into two stages: prefill processes prompts and is compute-intensive; decode generates output token by token and is memory-intensive. Disaggregated inference runs these two stages on separate chips. This means prefill chips may not need as much HBM, while decode chips need more—and racks often contain more decode chips than prefill chips. Whether total memory demand decreases or offsets depends on the specific ratio.

Chipmakers are already differentiating memory configurations based on workload. AMD's MI455X carries 432GB of HBM4, and Nvidia's Rubin carries 288GB. Nvidia also announced the Rubin CPX, a prefill chip using 128GB of GDDR7 instead of HBM. Startup Positron's Asimov chip skips HBM entirely, using LPDDR5X with configurable capacities from 288GB to 2.3TB, allowing customers to choose capacity based on workload.

Here arises a dimension often overlooked: not just total volume, but structure (mix). GDDR7 and LPDDR are still DRAM, and Micron sells both. However, their profit margins differ from HBM. Lyons cites Micron's own data: HBM consumes three times the wafers per bit of DDR5, and "this trade ratio only increases with future generations of HBM." Therefore, every bit migrated from HBM to LPDDR or GDDR due to architectural optimization releases a significant amount of DRAM wafers.

The key counter-intuitive judgment here: Lyons points out that HBM currently has lower margins than Micron's other DRAM products. Thus, bits migrating from HBM to LPDDR/GDDR might not be bad news; they could actually boost profits. This means simply tracking the total volume of HBM bits is insufficient to judge Micron's profitability trajectory; one must simultaneously track changes in demand structure.

Lyons concludes that memory demand now depends on how each accelerator is sized for its workload. Forecasting demand means tracking the direction of accelerator architecture—how much HBM does a training chip carry? Does prefill use HBM or GDDR? Is the KV cache placed in HBM, SRAM, or LPDDR? These architectural choices determine how many bits the industry needs and what kind of bits.

He also mentioned that Micron co-designs custom base dies with customers starting from HBM4E, so it should understand these trends more thoughtfully than the answer given at the meeting. Lyons believes Micron indeed has answers, they just haven't been fully articulated.

The second part of the article raises another question—who captures the value when customers put their own logic into the base die—but that section is paywalled and unreadable.

Implications for investors: When evaluating memory companies like Micron, accelerator architecture roadmaps should be treated as core variables for demand forecasting, rather than just looking at HBM shipment volumes or total DRAM bits. Architectural differentiation may lead to growing total bit volumes but improved margins, or flat total bit volumes but declining profits due to structural changes. This is a mechanism requiring continuous tracking, not a one-time event.

Sources

  • newsletter.chipstrat.com↗
Continue researching →