독서리서치레이더투자 프레임워크
로그인 / 회원가입
← 읽기로 돌아가기
뉴스주제 · NBER AI复现论文중대1 분 소요DisconfirmAI 편집부

AI produces 36 science manuscripts in three months, expert review still essential

A Harvard team used an open-source harness to run Claude across 18 fields; output was large but every result needed expert verification.

2026-10-01 08:00 GMT+8

Harvard physicist Matthew Schwartz used the open-source BootLoops harness to drive Claude, producing 36 manuscripts with 19 co-authors in three months, spanning 18 fields from particle physics to linguistics.

Concrete results include 30 elliptic integrals computed by Claude, fifteen of them for the first time, and a solution to a 20-year-old equation in neutral biodiversity theory that had been impossible to compute at scale; applied to data, it showed tree species composition on Barro Colorado Island in Panama changing 4.5 times faster than the theory allows.

In his guest post on Anthropic's site, Schwartz warns the model declares victory early, automated checks are unreliable, and conclusions can be wrong even when calculations are correct — each result became scientifically valuable only after domain experts stepped in. The manuscripts have not been peer reviewed.

출처

  • anthropic.com↗
주제

NBER AI复现论文

NBER工作论文W35782揭示,大语言模型(LLM)在批量复现经济学研究时,对近八成论文标记出结果出入。Matthew Schwartz、Isaiah Andrews与Jesse M. Shapiro开发的开源工作流,利用已发表的复现包对…

3 단계

로그인 / 회원가입
독서리서치레이더투자 프레임워크
2 건
업데이트됨 2026-10-06 23:11

현재까지의 과정

  1. 2026-09-29NBER发布工作论文W35782,介绍一种能复现、改进和扩展已发表经济学研究的LLM开源工作流。wesearch.press ↗
  2. 2026-10-01经济学家Tyler Cowen在Marginal Revolution撰文,详细披露该研究数据:4,452个复现包中3,460篇被标记结果出入。marginalrevolution.com ↗
  3. 2026-10-03Anthropic发布博客,作者Matthew Schwartz阐述“Claude-shaped science”理念,解释为何LLM在特定定量科学任务上高效但需专家把关。anthropic.com ↗

아직 해결되지 않은 문제

  • 被LLM标记为“结果出入”的3,460篇论文中,有多少比例经人工复核确认为真实错误?
  • 该开源工作流在非经济学领域的通用性及误报率如何?
  • 学术界是否会建立针对LLM辅助复现的标准验证协议?

이 스레드의 다른 내용

  • LLM reproduces economics papers at scale, with discrepancies flagged in nearly 80%2026-10-03
전체 주제 읽기 →
리서치 계속하기 →