ReadingResearchRadarInvestment framework
Sign in / Sign up
AllTechnologyProductsInvestingPublic marketsOther

Reading

2026-11-182 posts

Refresh

"Asynchronous Methods for Deep Reinforcement Learning": Replacing Experience Replay with Parallel Agents for Efficient Training on a Single CPU

Verified 2026-11-18 00:04 GMT+8

Long-term reading · 《Asynchronous Methods for Deep Reinforcement Learning》(2016)

In 2016, the Google DeepMind team published the paper "Asynchronous Methods for Deep Reinforcement Learning", introducing the A3C algorithm. Prior to this, deep reinforcement learning primarily relied on an "experience replay" mechanism to stabilize training, which involved storing historical agent data and randomly sampling updates. While effective, this approach required significant memory and computational resources and restricted the use to off-policy algorithms.

The core innovation of this paper was abandoning experience replay in favor of an asynchronous parallel architecture. By running multiple independent "actor-learner" threads on a standard multi-core CPU, each thread interacted with the environment and computed gradients independently. This parallelism naturally decoupled the temporal correlation of data, making the training process more stable while enabling the use of a broader range of online policy algorithms (such as Sarsa and n-step methods).

Experimental results showed that the best-performing Asynchronous Advantage Actor-Critic (A3C) method surpassed state-of-the-art approaches in Atari game domains. Notably, it achieved this using only a single multi-core CPU rather than a GPU, halving training time. This finding demonstrated that efficient deep reinforcement learning could be achieved without expensive hardware or distributed clusters, providing a lightweight design philosophy for subsequent algorithms.

Sources:《Asynchronous Methods for Deep Reinforcement Learning》(2016)

Research

Berkshire Hathaway 1996 Shareholder Letter: Intrinsic Value, Cost Efficiency, and Acquisition Standards

Verified 2026-11-18 00:04 GMT+8

Long-term reading · 《Berkshire Hathaway Shareholder Letter 1996》(1996)

In Berkshire Hathaway Inc.'s 1996 shareholder letter, Warren Buffett elaborated on the core investment philosophy of distinguishing book value from intrinsic value. Buffett noted that although the company's net assets and book value per share had grown, true economic progress is measured by intrinsic value rather than accounting figures. He illustrated this through GEICO's "step acquisition" case, explaining that accounting standards may require impairment of previously held shares, which does not reflect actual economic value appreciation. This commitment to intrinsic value distinguishes Berkshire from firms solely focused on short-term financial metrics.

The letter also emphasized alignment between management and shareholders' interests, particularly through cost control and avoiding dilution of shareholder equity. Buffett mentioned that Berkshire's headquarters expenses were extremely low, amounting to only two basis points of net assets, embodying the principle: "We are here to make money with you, not from you." He further discussed the relationship between stock price and intrinsic value, arguing that over the long term, shareholder returns must align with the actual growth of underlying businesses. When market prices temporarily deviate from intrinsic value, astute investors may exploit this gap for profit, while long-term holders should focus more on business fundamentals.

Regarding acquisition strategy, Buffett outlined Berkshire's distinctive criteria: seeking companies with excellent economic characteristics and outstanding management. Using Kansas Bankers Surety and FlightSafety International as examples, he demonstrated how personal networks can uncover opportunities and how transactions can prioritize cash over stock issuance to protect existing shareholders. These cases highlight both Berkshire's execution capability and its prudent approach to capital allocation.

For contemporary readers, this letter provides a clear framework for assessing a company's true value and management integrity. It reminds investors not to be misled by short-term market volatility or complex accounting adjustments, but to focus instead on long-term economic moats and management's capital allocation skills. The "Berkshire Hathaway 1996 Shareholder Letter," written by Warren Buffett and published on February 28, 1997 (covering fiscal year 1996), is suitable for investors seeking to deeply understand value investing principles, corporate governance, and M&A logic. Readers are advised to begin with sections on "The Relationship Between Intrinsic Value and Market Price" and "Acquisitions in 1996" to grasp Buffett's core ideas.

Sources:《Berkshire Hathaway Shareholder Letter 1996》(1996)

Research

You’re all caught up in this view

Sign in / Sign up
ReadingResearchRadarInvestment framework