LecturaInvestigaciónRadarMarco de inversión
Iniciar sesión / Registrarse
← Volver a la lectura
NoticiasTema · NBER AI复现论文Material1 min de lecturaEditorial de DisconfirmAI

AI produces 36 science manuscripts in three months, expert review still essential

A Harvard team used an open-source harness to run Claude across 18 fields; output was large but every result needed expert verification.

2026-10-01 08:00 GMT+8

Harvard physicist Matthew Schwartz used the open-source BootLoops harness to drive Claude, producing 36 manuscripts with 19 co-authors in three months, spanning 18 fields from particle physics to linguistics.

Concrete results include 30 elliptic integrals computed by Claude, fifteen of them for the first time, and a solution to a 20-year-old equation in neutral biodiversity theory that had been impossible to compute at scale; applied to data, it showed tree species composition on Barro Colorado Island in Panama changing 4.5 times faster than the theory allows.

In his guest post on Anthropic's site, Schwartz warns the model declares victory early, automated checks are unreliable, and conclusions can be wrong even when calculations are correct — each result became scientifically valuable only after domain experts stepped in. The manuscripts have not been peer reviewed.

Fuentes

  • anthropic.com↗
Tema

NBER AI复现论文

NBER工作论文W35782揭示,大语言模型(LLM)在批量复现经济学研究时,对近八成论文标记出结果出入。Matthew Schwartz、Isaiah Andrews与Jesse M. Shapiro开发的开源工作流,利用已发表的复现包对…

Iniciar sesión / Registrarse
LecturaInvestigaciónRadarMarco de inversión

3 pasos2 piezasActualizado 2026-10-06 23:11

Cómo llegó hasta aquí

  1. 2026-09-29NBER发布工作论文W35782,介绍一种能复现、改进和扩展已发表经济学研究的LLM开源工作流。wesearch.press ↗
  2. 2026-10-01经济学家Tyler Cowen在Marginal Revolution撰文,详细披露该研究数据:4,452个复现包中3,460篇被标记结果出入。marginalrevolution.com ↗
  3. 2026-10-03Anthropic发布博客,作者Matthew Schwartz阐述“Claude-shaped science”理念,解释为何LLM在特定定量科学任务上高效但需专家把关。anthropic.com ↗

Lo que aún está pendiente

  • 被LLM标记为“结果出入”的3,460篇论文中,有多少比例经人工复核确认为真实错误?
  • 该开源工作流在非经济学领域的通用性及误报率如何?
  • 学术界是否会建立针对LLM辅助复现的标准验证协议?

También en este hilo

  • LLM reproduces economics papers at scale, with discrepancies flagged in nearly 80%2026-10-03
Leer el tema completo →
Continuar investigando →