読書リサーチレーダー投資フレームワーク
ログイン / 新規登録
← 読解に戻る
ニューストピック · NVIDIA中间训练配方重大1 分で読めるDisconfirmAI 編集部

NVIDIA Releases Open Midtraining Recipe, Claims 3.8-Point Gain on SWE-bench for Nemotron Models

NVIDIA publishes an open recipe for the 'midtraining' stage between pretraining and fine-tuning, reporting significant self-tested gains in coding agent benchmarks.

検証済み 2026-10-17 01:23 GMT+8

NVIDIA researchers have published an open recipe for "midtraining" on arXiv, addressing the data gap between pretraining and post-training (specifically Supervised Fine-Tuning or SFT) in frontier models. The study notes that pretraining typically uses short contexts with little agent interaction, while SFT relies mostly on synthetic data and can harm later RL trainability if too long. Midtraining fills this gap by training at 256K context length on a mixture of natural software-engineering data (repositories, PRs, commit histories) and a small share of agentic traces.

According to the paper's abstract, on the Nemotron 3.5 Lightning model, 500 billion midtraining tokens improved the SWE-bench Verified score from 60.7 to 64.5, and SWE-bench Multilingual from 55.8 to 62.8. On the larger Nemotron 3.5 Super, 280 billion tokens improved DeepSWE from 11.7 to 15.7. These results were achieved after identical full SFT runs.

The authors also report an evaluation protocol based on short fixed SFT runs, noting that base-model proxy metrics did not reliably correlate with post-training results. These are first-party benchmark claims by NVIDIA and have not yet been independently reproduced; the findings are specific to the Nemotron model family.

ソース

  • arxiv.org↗
トピック
ログイン / 新規登録
読書リサーチレーダー投資フレームワーク

NVIDIA中间训练配方

NVIDIA发布开源“中间训练”(Midtraining)配方,旨在填补大模型预训练与后训练之间的空白,提升代码智能体性能。该阶段在256K上下文长度下,使用数百亿个自然软件工程数据Token进行训练。实验显示,在Nemotron 3.5 …

2 ステップ0 件更新日 2026-10-18 09:47

経緯

  1. 2026-10-15NVIDIA团队在arXiv提交论文《Priming Base Models for Agentic Post-Training: An Open Midtraining Recipe》,公开中间训练的数据构造方法与消融实验结果。arxiv.org ↗
  2. 2026-10-17媒体广泛报道该配方,强调其在Nemotron系列模型上带来的SWE-bench等代码基准分数显著提升,以及其对Agent鲁棒性的改善。aiweekly.co ↗

未解決の問題

  • 该中间训练配方在非Nemotron架构(如Qwen、GLM)上的迁移效果如何?
  • 自然软件工程数据与少量Agent轨迹的最佳混合比例在不同规模模型中是否一致?
  • 社区复现该配方时,所需的计算资源门槛是否会限制中小实验室的应用?
トピック全文を読む →
リサーチを続ける →