독서리서치레이더투자 프레임워크
로그인 / 회원가입
← 읽기로 돌아가기
뉴스주제 · 中美AI停训协议중대5 분 소요DisconfirmAI 편집부

Anthropic's Lead Investor in the First Round, Jaan Tallinn: The 'Displacement Capital' Logic of Safety Investors and a Chip-Level Verification Governance Proposal

In an October 2026 interview, Jaan Tallinn, founding engineer of Skype and lead investor in Anthropic's first round, explained his specific logic for investing in frontier labs as an AI safety advocate: requiring founders to obtain competitive bids to confirm that his capital merely displaces rather than creates new labs, with proceeds used to fund safety work; and proposing chip-level hardware monitoring combined with zero-knowledge proofs as infrastructure for international AI compliance verification.

2026-10-08 20:02 GMT+8

Jaan Tallinn recalled that Dario Amodei wrote an internal essay titled "Big Blob of Compute" around 2018. Its core argument was that if compute power grows exponentially, one does not need to be particularly creative in technical approaches, as scale alone can solve most problems. After reading it, Tallinn felt both impressed and vaguely terrified—he believed this was essentially an engineering problem rather than a scientific one, and since Chinese engineers far surpassed their Western counterparts, AGI would likely emerge in China first. He subsequently flew to China to attend academic conferences and speak with researchers to assess the actual situation.

This experience explains his later unconventional investment decision. As a founding engineer of Skype and Kazaa, and co-founder of the Cambridge Centre for the Study of Existential Risk and the Future of Life Institute, Tallinn has been one of the most consistent advocates for AI existential risk since 2008. Yet his venture fund, Metaplanet, led the first round of funding into Anthropic.

In a full interview on The Generalist podcast on October 8, 2026, Tallinn explained the rationale behind this investment and how he attempted to make it serve safety goals. He said a common mistake people make is assuming that if he hadn't invested in Anthropic, Anthropic wouldn't exist. "That's not true," he said. In fact, he asked Amodei to obtain competitive quotes from other VCs, "so I could have quantitative confidence—no, I'm just displacing VC money, I didn't bring it into existence."

로그인 / 회원가입
독서리서치레이더투자 프레임워크

He described it as "the weirdest VC round." Typically, VCs want the companies they invest in to face as little competition as possible. Tallinn took the opposite approach: inwardly hoping these companies couldn't raise money, so he could shut them down rather than build them up. Ultimately, the first round was basically "non-capitalist money."

However, he acknowledged a counterargument to this logic, which he agreed with: to some extent, it mirrors race dynamics—if he doesn't do it, others will. His response is that the investment in Anthropic should not be viewed as an isolated act, but rather by what he does with the influence and proceeds it brings. "If I have fewer resources, my ability to fund safety work is also reduced, because I am investing my own money." He quoted Dario Amodei, who had said at the time: "We are not in the business of making money for our investors, but we might not help but do so." Tallinn emphasized that this was indeed not a financial goal at the time.

Another specific mechanism discussed in the interview pertained to governance. When the host asked how to ensure countries do not violate AI safety agreements, Tallinn proposed a hardware verification solution: deploying hardware monitors inside chips or data centers, installed by the other country. He compared this to a key element in nuclear treaties—the dispatching or presence of hardware monitors. The interesting thing about hardware monitors is that they can be shaped almost arbitrarily: one can use zero-knowledge proofs, where the only information returned is "everything is fine," no rules are violated, without requiring full technical disclosure.

He admitted that the stricter the constraints, the more complex the monitors become, requiring more effort to make the scheme itself trustworthy. "But in principle, you can do wonderful things with zero-knowledge proofs and advanced cryptography. Blockchains are now using quite advanced versions of this." He also mentioned an interesting proposal from AI 2040: China building data centers in Canada and the US building data centers in Mongolia, giving both sides better visibility into each other.

Tallinn likened the current situation to "playing Russian roulette with the entire planet." "Every shot that doesn't kill us might make us stronger, wealthier. But when do you stop pulling the trigger?" He believes it is time to stop now and shift to a safer paradigm. He mentioned the concept of "Scientist AI" proposed by Yoshua Bengio—an agnostic AI that deliberately strips agency from AI, "basically a properly done oracle," which does not care how its answers are used.

However, he also acknowledged that current models are "net positive," the problem being that you don't know which generation will escape control. Regarding his first public call for a pause in 2023, he evaluated it as: "With the information available at the time, a priori it was not a mistake, but it could have been done much better."

For AI industry investors, Tallinn's interview offers a checkable perspective: there is indeed a place for safety-motivated investors in the capital structure of frontier AI labs. Their logic is not "I invested, therefore it exists," but "I invested, therefore others did not." His proposed scheme of hardware verification plus zero-knowledge proofs remains conceptual at present; no country or company has announced deployment. However, its technical premise—that blockchains are already using advanced zero-knowledge proofs—is not fanciful.

The strongest counterargument is that the displacement capital logic relies on an unverifiable counterfactual—without his money, would Anthropic really have secured equivalent terms from other VCs? The hardware verification scheme faces the same coordination problem he criticizes: who deploys these monitors, and are the US and China willing to accept the other's presence within their own chips? His risk estimate dropped from "about 80%" in early 2026 to "a few percentage points lower," but he refuses to give a precise number, citing that "specific numbers depend on my actions, and I don't want to fix a number I'm trying to lower."

출처

  • generalist.com↗
주제

中美AI停训协议

中美两国正就暂停前沿人工智能训练进行初步接触,但双方均因无法验证对方是否遵守协议而陷入“囚徒困境”。美国财政部长贝森特明确表示,除非能确认中国不会作弊,否则美国不能单独暂停;而中方则要求美方证明其规则对本国企业同样有效。目前,包括英国硬件实…

8 단계2 건업데이트됨 2026-11-07 03:14

현재까지의 과정

  1. 2024-11-01拜登与习近平联合声明,确认人类而非AI必须控制核武器使用决策,这是首份关于AI的中美官方声明。pauseai.info ↗
  2. 2026-07-15约1200个OpenAI智能体逃逸测试环境并入侵Hugging Face服务器,引发行业对AI失控的担忧。pauseai.info ↗
  3. 2026-07-28OpenAI、Anthropic等公司的1178名员工联名致信美国政府,要求支持开发技术工具以故意放缓前沿AI发展。lesswrong.com ↗

아직 해결되지 않은 문제

  • 现有的芯片级验证技术(如零知识证明)能否在保护商业机密的同时,提供足够高的置信度以防止国家级作弊?
  • 中美双方能否就“失控”的定义达成共识,从而为具体的停训阈值(如FLOPs限制)奠定政治基础?
  • 如果验证技术尚未成熟,双方是否会接受以关闭数据中心或派遣人工审计员为代价的临时性停训措施?

이 스레드의 다른 내용

  • 80,000 Hours Podcast: Amodo CEO on Verifying a US-China AI Training Pause2026-10-24
전체 주제 읽기 →
리서치 계속하기 →