読書リサーチレーダー投資フレームワーク
ログイン / 新規登録
← 読解に戻る
ニューストピック · NBER AI复现论文重大1 分で読めるDisconfirmAI 編集部

LLM reproduces economics papers at scale, with discrepancies flagged in nearly 80%

Economists used an LLM to reproduce economics research in bulk, flagging discrepancies in 3,460 of 4,452 replication packages; flagged results are automated detections, not proof of error.

検証済み 2026-10-03 10:51 GMT+8

Readers can now reproduce published economics research in bulk with a large language model: of 4,452 published replication packages, 3,460 were flagged for discrepancies, while calculations in 496 papers were sped up by more than a factor of 10 and 923 received extensions consistent with the original aims.

Previously, such reproduction relied on manual, paper-by-paper checks, which were costly and limited in coverage.

The workflow was built by economists Matthew Schwartz, Isaiah Andrews and Jesse M. Shapiro. It first reproduces the original calculations, then checks them against published findings and runs sensitivity analysis; the flagged counts and speedups are author-reported.

The flagged "discrepancies" are the workflow's automated detections, not proof the original papers were wrong; the paper has not been peer reviewed, is NBER working paper w35782, was covered by Tyler Cowen on Marginal Revolution on October 1, and has no third-party replication yet.

ソース

  • nber.org↗
ログイン / 新規登録
読書リサーチレーダー投資フレームワーク
marginalrevolution.com
↗
トピック

NBER AI复现论文

NBER工作论文W35782揭示,大语言模型(LLM)在批量复现经济学研究时,对近八成论文标记出结果出入。Matthew Schwartz、Isaiah Andrews与Jesse M. Shapiro开发的开源工作流,利用已发表的复现包对…

3 ステップ2 件更新日 2026-10-06 23:11

経緯

  1. 2026-09-29NBER发布工作论文W35782,介绍一种能复现、改进和扩展已发表经济学研究的LLM开源工作流。wesearch.press ↗
  2. 2026-10-01经济学家Tyler Cowen在Marginal Revolution撰文,详细披露该研究数据:4,452个复现包中3,460篇被标记结果出入。marginalrevolution.com ↗
  3. 2026-10-03Anthropic发布博客,作者Matthew Schwartz阐述“Claude-shaped science”理念,解释为何LLM在特定定量科学任务上高效但需专家把关。anthropic.com ↗

未解決の問題

  • 被LLM标记为“结果出入”的3,460篇论文中,有多少比例经人工复核确认为真实错误?
  • 该开源工作流在非经济学领域的通用性及误报率如何?
  • 学术界是否会建立针对LLM辅助复现的标准验证协议?

このスレッド内の他の記事

  • AI produces 36 science manuscripts in three months, expert review still essential2026-10-03
トピック全文を読む →
リサーチを続ける →