Reading

When traffic peaks arrive, text-to-speech (TTS) requests begin queuing, and latency breaches the service level objective (SLO)—even though auto-scaling policies were supposed to prevent this. The problem lies in the fact that the thresholds these policies rely on are often guessed based on experience. If guessed too conservatively, GPU instances sit idle, costing money on every invoice without triggering any alerts. If guessed too aggressively, each instance carries more concurrent requests than its latency promise allows. Everything looks normal in the console until a real traffic spike pushes queueing time past the SLO, exposing the failure at the moment most callers are listening.

Why can't TTS use the same scaling metrics as large language models? Most public guidelines target short synchronous request/response calls or LLM inference, where the natural metric is tokens generated per second. However, non-autoregressive models like Kokoro-82M produce full audio from a single forward pass, lacking the token-by-token generation loop needed to attach such metrics. What determines whether a TTS instance is overloaded is simpler: how many requests are currently queued and executing. Therefore, this solution uses SageMaker AI's native ConcurrentRequestsPerModel metric instead of custom throughput measures.

The core method is called the four-number model: operators select two parameters—the latency SLO (3,000 ms p95 time-to-first-audio-byte in this case) and an overflow ratio (1.25x, allowing traffic to grow by 25% within one scaling cycle without breaching the SLO)—then perform two measurements on the live endpoint. First is Q_max: using a closed-loop load ladder, increase concurrency step-by-step until p95 latency breaches the SLO, taking the last concurrency value that met the SLO. For Kokoro-82M on ml.g5.xlarge, Q_max was measured at 50. Second is T_total: force a scale-out event, timing each stage from capacity request to healthy traffic recovery. In this case, pure provisioning took 260.2 seconds; adding the upper bound of policy detection lag (60 seconds), the planned T_total is 320.2 seconds.

With Q_max and T_total, scaling thresholds and cooldowns are derived arithmetically: Scale-out threshold C_scale_max = (1−h) × Q_max = 0.75 × 50 = 37.5; Scale-in threshold C_scale_min = (1−2h) × Q_max = 0.50 × 50 = 25. The scale-out cooldown is short (30 seconds) because the cost of adding one instance is low. The scale-in cooldown is long (approximately 3 times T_total, i.e., 960 seconds) because if traffic rises again after removing an instance, re-scaling requires the full 320 seconds. The container’s own admission control hard-limits queue depth at Q_max=50, returning a 503 rejection for excess requests rather than letting them queue to inevitable timeout.

The key insight of this method extends beyond TTS. The blog notes that for latency-sensitive, concurrency-bound real-time inference endpoints on SageMaker AI—such as an image embedding endpoint keeping connections open during GPU forward passes—the bottleneck is also concurrent in-flight requests, not throughput metrics borrowed from other workload shapes. The derivation logic and five-step toolchain (drift, qmax, ttotal, plan, validate) can be migrated directly. However, specific values differ by model and instance type: Q_max=50 and T_total=320 seconds apply only to Kokoro-82M on ml.g5.xlarge; different models or machine types require new measurements.

Ben Carlson wrote on his blog "A Wealth of Common Sense" that a reader, aged 38 with all assets in broad-based index funds, asked him: "I recently heard you bought bonds for the first time in your career; how did you decide which type to buy?" His answer was not "because interest rates are going to fall," but rather a complete line of reasoning extending from duration sensitivity to starting yields to life-stage risk budgets.

Duration is the core concept of this article. Carlson explains that duration is essentially interest rate sensitivity—a bond with a duration of 5 years will see its price drop by approximately 5% for every 1 percentage point rise in interest rates; conversely, if interest rates fall by 1 percentage point, the price will rise by approximately 5%. He gives an example that the U.S. long-term Treasury ETF (TLT) currently has a duration of about 15 years, meaning a 1 percentage point rise in interest rates would result in a price loss of approximately 15%. "I don't like guessing interest rate trends," he wrote, "I prefer holding intermediate-to-short term durations of 5 to 6 years or shorter."

Behind this choice lies a principle he repeatedly emphasizes: only bear volatility when paid a risk premium. The volatility of long-term Treasuries is too large, so holding them is essentially speculating on the direction of interest rates rather than collecting fixed income. Intermediate-to-short term bonds provide a relatively certain source of return—"collecting interest"—while keeping the price impact caused by interest rate changes within an acceptable range.

So why now? The direct reason Carlson gives is the starting yield. "Bond returns tend to track starting bond yields," he wrote, "Expected returns for bonds are higher than they have been at any time since Apple released its first iPhone." For the past twenty years—from the ultra-low interest rates after the U.S. financial crisis to further suppression during the pandemic—fixed income provided almost no decent income. Now yields have returned to their highest levels since 2007, changing the math.

But he simultaneously incorporates personal life stage into the decision. "I am also at a different stage in my life; I am middle-aged, and my goals are different." He bought some tax-efficient bonds for his brokerage account, which might be used for vacations, home renovations, ammunition for the next stock market downturn, or simply to "collect 5.5% interest and be happy." He still holds significant equity risk in both his retirement accounts and his brokerage account.

Finally, he clearly defines what he is NOT doing: "This is not predicting a stock market crash; the bull market could certainly continue. This is not predicting an interest rate collapse; if inflation remains sticky, the U.S. government continues spending, and AI capital expenditures soar, bond yields could absolutely continue to rise." He admits that if interest rates rise, his bond prices will fall in the short term, but believes that the current higher yields offer a "margin of safety" that hasn't existed for some time—and if interest rates do continue to rise, it means that once the dust settles, expected returns will be even higher.

"Or maybe I'm just getting old and boring," he wrote, "but either way, bond yields look attractive for the first time in a long time."

The exponential growth of AI compute rests on a physical prerequisite: someone must manufacture the chips. Intel Chief Technology Officer Pushkar Ranade provided a specific figure during an episode of the Dwarkesh Patel podcast aired on October 22: even with unlimited capital, the global stock of existing chipmaking tools can at most double leading-edge wafer production. Meanwhile, the AI industry’s compute growth rate over the past decade has been threefold per year.

What does this gap between these two figures imply? Ranade’s answer is that output cannot increase unless the entire supply chain—from equipment suppliers to their suppliers and sub-suppliers—produces more. He specifically highlighted that ASML produces fewer than 100 EUV lithography machines annually, with no plans to significantly exceed this number by the end of the current decade. These machines must be allocated among Intel, TSMC, and all memory manufacturers.

The deeper constraints extend beyond EUV. Ranade pointed out that once a process flow is locked in—for example, the now-mature 18A node—specific tools are required, and one cannot simply switch to equipment from another supplier. Even if multiple suppliers exist for certain tools, process lock-in means reliance on a single source remains effectively true. Equipment delivery cycles are at least 18 months: placing massive orders today means receiving equipment no earlier than 2028; if new fab shells need to be built as well, that pushes delivery to the end of the decade.

This does not mean capacity growth is impossible. Ranade distinguished between two dimensions: marginal output can be increased on a 12-month scale (by optimizing utilization of existing equipment), but total volume growth through the end of the decade will be extremely limited. Intel’s own experience corroborates this: moving from 18A to 14A allows reuse of approximately 80% of the toolset, but denser geometries require new equipment.

So what drives continued AI compute growth? In the interview, Ranade and Chris Auth, who leads development of all Intel’s mass-production processes, offered another path: making each chip on every wafer more powerful. Auth described the evolution from planar transistors to FinFETs to gate-all-around architectures, and the next step of CFET (Complementary Field-Effect Transistors, which stack n-type and p-type transistors). However, Auth admitted their visibility into technology roadmaps extends only about four years; predicting specific forms beyond a decade is difficult.

What does this constraint mean for AI readers? It indicates that AI compute’s exponential growth is not automatic. Hardware capacity has a hard upper limit determined by the equipment supply chain. Breaking through it requires either continuous improvement in per-chip performance (new architectures, new materials) or synchronized expansion across the entire supply chain (which itself is constrained by capital, time, and physical conditions). The 2X cap cited by Intel’s CTO is the most direct quantitative expression of this hard constraint.

Travis Knight, CEO of stop-motion animation studio Laika and director of the new film "Wildwood," drew a clear distinction between pencils, paintbrushes, 3D printers, and workstations, and generative AI during an interview with The Verge: the former have meaning and value when placed in the hands of an artist, while the latter represent 'technology doing the artist's work.' He stated his firm opposition to the latter, explaining that 'when technology consumes the material that artists truly create and then regurgitates it in a different way, there is no soul, no spirit, no humanity behind it—that is theft.' He further elaborated that only those who have experienced love, loss, and heartbreak can tell stories about these experiences, noting that computers have never had such experiences. Knight simultaneously acknowledged that AI may bring great benefits in the fields of medicine and science, making it clear that he is not opposed to all AI applications, but rather concerned about how it is being used.

In 2016, the Google DeepMind team published the paper "Mastering the game of Go with deep neural networks and tree search" in Nature magazine, introducing the AlphaGo program. This research addressed the classic AI challenge of Go by proposing a new method combining "value networks" and "policy networks." These deep neural networks were trained through supervised learning from human expert games and reinforcement learning from self-play.

The core innovation of the paper lies in combining Monte Carlo Tree Search (MCTS) with neural networks. The policy network is used to select move positions, while the value network evaluates board states. Without lookahead search, the neural networks alone achieved the level of state-of-the-art simulation-based random self-play programs at the time. With the introduction of the new search algorithm, AlphaGo achieved a 99.8% win rate against other Go programs and defeated European Go champion Fan Hui 5-0. This was the first time a computer program defeated a professional human player in a full-sized game of Go, an achievement previously thought to be at least ten years away.

For modern readers seeking to understand the origins of current large model reasoning capabilities, the architectural ideas in this paper are relevant. For example, consider a complex decision system that no longer relies solely on rule engines or simple statistical probabilities but, like AlphaGo, uses neural networks to "intuitively" filter out a few most promising action options (policy network), then performs deeper simulation evaluations on these options to judge the final outcome (value network). This hierarchical processing mechanism of "fast filtering + deep evaluation" is one of the key approaches to solving high-dimensional space search problems.

Recommended reading: "Mastering the game of Go with deep neural networks and tree search" (2016). For researchers and engineers interested in how deep learning applies to complex games and how prior knowledge of neural networks guides search algorithms, this is a foundational document. It is suggested to start with the abstract and Figure 1 (neural network training flow), focusing on the role of policy and value networks in node expansion within Monte Carlo Tree Search.

In the 1993 shareholder letter, Warren Buffett delves into the distinction between book value and intrinsic value and their significance for Berkshire Hathaway. He points out that while book value is an accounting term measuring capital invested in a business, what truly matters is per-share intrinsic value, which is the estimated present value of cash extractable from the business over its remaining life. For most companies, these two figures are unrelated, but Berkshire is an exception. Although its book value is significantly lower than its intrinsic value, it remains a useful tool for tracking this key metric. In 1993, both metrics grew by approximately 14%, which Buffett described as "satisfying but not exciting."

Furthermore, Buffett elaborates on Berkshire's unique capital allocation philosophy. He explicitly states that the company has no grand vision regarding which businesses or industries it will enter in the future, believing that such new businesses based on grand visions are usually harmful to shareholders of large enterprises. Instead, they focus on the economic characteristics of the businesses they wish to own and the personal characteristics of the managers they wish to work with. This philosophy was exemplified in the acquisition of Dexter Shoe. Dexter Shoe is a company located in Maine, USA, managed by Harold Alfond and Peter Lunder, who made their US factories highly competitive through clever management and a skilled workforce amidst competition from low-wage countries importing footwear. Buffett emphasizes that they chose Dexter because of its excellent management team and favorable economic characteristics, rather than due to any strategic plan.

For investors, understanding this distinction is crucial. The short-term market is a voting machine, but the long-term market is a weighing machine. Stock prices may deviate from intrinsic value, but they will eventually revert. Buffett's approach reminds us that investment should focus on the essential quality of the business and the capability of its managers, rather than chasing macro narratives or industry trends.

It is recommended to read the "Berkshire Hathaway 1993 Shareholder Letter," particularly the sections defining intrinsic value and the case study of the Dexter Shoe acquisition, to gain a deeper understanding of how Buffett balances financial metrics with management quality in capital allocation.