Reading

On October 15, 2026, BG2 Pod published an episode featuring NVIDIA CEO Jensen Huang alongside hosts Brad Gerstner and Clark Tang. This marks their third conversation, recorded shortly after significant events including the Xi State Dinner and the White House Accord on superintelligence.

The publisher's show notes indicate that Huang addresses core questions about AI economics, specifically whether current revenues can justify massive capital expenditures. He also outlines the biggest bottlenecks expected in 2027, explores the path toward singularity, and discusses the strategic differences between open and closed models.

Additional topics listed include the relationship between responsible technology building and speed, how power constraints might shape AI's next phase, and the possibility of SpaceX helping to place data centers in orbit. Huang is also noted as explaining why NVIDIA sees demand extending beyond just two major AI labs.

In the development of large language models, the training costs of dense models rise sharply as parameter counts increase, making simple model scaling unsustainable due to hardware resource limitations. Mixture-of-Experts (MoE) models, as a sparse architecture, can significantly reduce computational costs during training; however, their massive parameter count leads to a surge in memory requirements and low inference efficiency, limiting their practical application. The DeepSpeed-MoE paper addresses this pain point by proposing an end-to-end solution aimed at balancing model quality, training cost, and inference efficiency.

One of the core contributions of this research is expanding the scope of MoE applications and proposing new architectural designs. Traditional MoE models are mostly used for encoder-decoder tasks, but this paper successfully applies them to autoregressive natural language generation (NLG) tasks, demonstrating that a 5x reduction in training cost can be achieved while maintaining equivalent model quality. To address the issue of low "parameter efficiency" in MoE models, the authors designed Pyramid-Residual MoE (PR-MoE). This is a hybrid dense-sparse architecture that applies sparsity only where experts are most effective through residual connections, thereby reducing the parameter size of MoE models by up to 3x without changing model quality or significantly altering computational requirements. Furthermore, using staged knowledge distillation techniques, the authors created a distilled version of PR-MoE called Mixture-of-Students (MoS), which further compresses the model size to 1/3.7 of the original while maintaining comparable model quality.

Beyond architectural innovations, DeepSpeed-MoE also developed a highly optimized inference system to solve deployment challenges for large-scale MoE models. Since MoE models are much larger than equivalent dense models, inference processes are often limited by memory bandwidth rather than compute capacity. By combining model compression with system-level optimizations, this system achieves significant inference acceleration and cost reduction. Specifically, compared to existing MoE inference solutions, DeepSpeed-MoE provides up to 7.3x improvements in latency and cost; compared to dense models of equivalent quality, its inference speed is improved by 4.5x and costs are reduced by 9x. This enables ultra-low-latency inference below 25 milliseconds for trillion-parameter-level MoE models, providing a feasible engineering path for transitioning from dense models to sparse MoE models.

For readers interested in infrastructure optimization for large models, DeepSpeed-MoE offers a clear perspective on how to break through compute bottlenecks via hardware-software co-design. It is not merely an algorithmic improvement but a complete system solution, demonstrating how to deploy high-quality large models under resource constraints. For example, if a team needs to deploy a high-performance dialogue model on a limited GPU cluster, directly adopting a dense model might fail due to insufficient VRAM or excessive inference latency. However, by referencing the PR-MoE architecture and inference optimization strategies in DeepSpeed-MoE, they could potentially maintain response speed while significantly lowering hardware barriers.

Engineers and researchers engaged in AI systems engineering, model compression, and efficient inference research are recommended to read this paper, "DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale" (2022), focusing particularly on Section 1 regarding the description of challenges and Section 3 regarding technical details of PR-MoE and the inference system.

In his 1987 letter to shareholders, Warren Buffett delves into the distinction between book value and business value, articulating Berkshire Hathaway's core investment philosophy. First, he explicitly states: "What counts, of course, is the rate of gain in per-share business value, not book value." To illustrate this point, he cites examples such as LTV and Baldwin-United, which displayed high book values prior to their bankruptcies, and Belridge Oil, which was sold to Shell at a price far exceeding its book value. This serves as a reminder to investors that accounting figures often fail to reflect a company's true economic condition; real value lies in the enterprise's ability to generate cash and its competitive moat.

Second, Buffett emphasizes the value of operational stability, opposing the blind pursuit of so-called "high-growth stories." He writes: "Experience, however, indicates that the best business returns are usually achieved by companies that are doing something quite similar today to what they were doing five or ten years ago." He argues that while frequent major changes may sound appealing, they also create opportunities for significant errors and make it difficult to establish durable business franchises. Citing research from Fortune magazine, he notes that only a few companies maintain extremely high return on equity over a decade, and most of these operate in seemingly mundane businesses—products or services that are not glamorous but are executed exceptionally well.

Finally, the letter showcases Berkshire's unique decentralized management system. Buffett mentions: "With managers like ours, my partner, Charlie Munger, and I have little to do with operations. In fact, it is probably fair to say that if we did more, less would be accomplished." This trust stems from identifying excellent managers, such as those leading subsidiaries like Buffalo News and See's Candies. They not only generated astonishing capital returns but also allowed headquarters to focus on capital allocation rather than micromanagement. For today's readers, this provides a clear framework: when evaluating a company, one should focus on the durability of its business model and the autonomy of its management, rather than short-term financial embellishments or aggressive strategic pivots. If you are analyzing a traditional consumer goods company whose product lines and profit models have remained stable over the past decade, and whose management enjoys high autonomy, it may possess greater long-term investment value than a tech company constantly launching new concepts but lacking a track record of execution.

For investors seeking to understand how to see through financial statements to grasp the essence of a business, and how to construct a decentralized portfolio driven by outstanding managers, the Berkshire Hathaway Shareholder Letter 1987 is a must-read classic. It clearly defines the difference between business value and book value, and argues for an investment logic where stability outweighs change. It is recommended to start reading from the opening paragraphs regarding the definition of value, then focus on the description of subsidiary operating models.