ReadingResearchRadarInvestment framework
Sign in / Sign up
← Back to reading
News3 min read·2026-10-21DisconfirmAI Editorial

Chi Nguyen Explains Updatelessness: Why Agents Should Follow the Ex Ante Optimal Policy

Chi Nguyen published a long post on LessWrong using the calculator bet and the Sleeping Beauty variant to show that when an agent's policy affects other instances or counterfactual versions of itself, standard Bayesian updating leads to double-counting or neglecting counterfactual worlds, producing systematic errors. The updateless strategy requires an agent to always act according to the optimal policy from the perspective before observing the current situation, but this is only meaningful under specific structural conditions, and determining the prior remains an unsolved problem.

ImportanceMaterialEvidenceE3 inspectableWrite-upStandard

Original publication time 2026-10-21 06:28 GMT+8

An agent faces the following bet: there are 100 copies of you in the universe, your calculator judges the parity of the 1,000,000th digit of pi with 99% accuracy, and it says the digit is even. The bet rules are: if the digit is indeed even, you win $1; if it is odd, you lose $1,000. You care about the total payoff across all copies, and you know you were only offered this bet because your calculator outputted "even."

In a long post published on LessWrong on October 20, 2026, Chi Nguyen points out that intuitively this bet should not be accepted: the ex ante expected value is 0.5 × 99 × $1 − 0.5 × 1 × $1,000 = −$450.50. But an evidential decision theory agent using standard Bayesian updating will accept it.

Where does the error come from? The updated agent observes that the calculator says "even" and updates her credence to 99% that she is in the even-world. She then reasons: in the even-world, about 99 copies see the same output, and my choice gives them evidence, so accepting the bet means 99 copies each win $1, totaling $99. In the odd-world, only 1 copy sees "even," and accepting the bet means losing $1,000. Weighted expectation: 0.99 × $99 − 0.01 × $1,000 = $88.01, which is positive, so she accepts.

Sign in / Sign up
ReadingResearchRadarInvestment framework

Nguyen identifies the problem: the factor of 99 is double-counted—it appears once in the 99:1 credence update and again in the 99-copy payoff in the even-world. The updateless solution is to stick with the prior odds of 1:1 rather than the updated 99:1, giving an expected value of 0.5 × $99 − 0.5 × $1,000 = −$450.50, so the agent correctly declines.

This leads to the core definition of updatelessness: an updateless agent follows the policy she would have wanted to commit to before knowing which situation she is in, even if a different action seems best after updating on her observations. Nguyen distinguishes two levels: pseudo-updatelessness (achieved through prior commitment, which both EDT and CDT recommend when commitment is possible) and full updatelessness (requiring no prior commitment, applying even to situations the agent is born into).

The calculator bet illustrates why updating less is helpful, but the author explicitly notes it does not require full updatelessness—combining EDT with a specific updateful theory of anthropics (mrcSSA) also gets the right answer. What truly requires full updatelessness is the Sleeping Beauty variant. Similarly, in the "born into" version of XOR blackmail, Omega decides whether to create the agent based on her policy; the agent cannot commit in advance, but updateless reasoning still gives the correct result.

But Nguyen also gives clear boundary conditions. He cites MacAskill's "hummus vs. avocado" example to show that in most mundane decisions, the ex ante optimal policy is simply to choose according to current preferences, and updateless and updateful behavior are identical. Updatelessness is only meaningful under special structural conditions—namely, when the output of one's policy on one observation changes what happens under another observation or the probability of each observation. In the calculator bet, the policy affects other copies' behavior; in XOR blackmail, the policy affects the probability of receiving the letter.

The article concludes by acknowledging an unsolved problem: in toy thought experiments, the ex ante perspective and prior are well-defined, but in real life an agent must determine her prior somehow. Nguyen writes that how to do this is a difficult question and outside the scope of the post. This means updatelessness provides a principled direction, but practical deployment still requires solving the prior specification problem.

Sources

  • 原始出处lesswrong.com↗
Continue researching →