読書リサーチレーダー投資フレームワーク
ログイン / 新規登録
← 読解に戻る
ニューストピック · NVIDIA中间训练配方重大1 分で読めるDisconfirmAI 編集部

NVIDIA Releases Open Midtraining Recipe, Reports Gains in Coding Agent Benchmarks

NVIDIA publishes a recipe bridging pretraining and fine-tuning with real software data, reporting significant gains in coding agent benchmarks.

2026-10-16 12:00 GMT+8

NVIDIA researchers submitted an arXiv preprint on October 15 detailing an open "midtraining" recipe for large language models. The method aims to bridge the gap between short-context pretraining and synthetic-data supervised fine-tuning (SFT) by incorporating vast amounts of natural software engineering data to enhance coding agent capabilities.

The recipe trains models at a 256K context window using a mixture of packed repositories, pull requests, commit histories, build logs, and a small share of agentic traces. On NVIDIA's own Nemotron 3.5 Lightning model, 500 billion midtraining tokens improved the SWE-bench Verified score from 60.7 to 64.5, and SWE-bench Multilingual from 55.8 to 62.8. On the larger Nemotron 3.5 Super, the DeepSWE benchmark score rose from 11.7 to 15.7.

The study notes that traditional pretraining rarely sees agent interactions, while purely synthetic SFT data can hurt later RL trainability. Midtraining addresses this by mixing real-world code data. These results are currently self-reported by the authors and have not yet been independently reproduced.

ソース

  • arxiv.org↗
トピック

NVIDIA中间训练配方

NVIDIA发布开源“中间训练”(Midtraining)配方,旨在填补大模型预训练与后训练之间的空白,提升代码智能体性能。该阶段在256K上下文长度下,使用数百亿个自然软件工程数据Token进行训练。实验显示,在Nemotron 3.5 …

ログイン / 新規登録
読書リサーチレーダー投資フレームワーク

2 ステップ0 件更新日 2026-10-18 09:47

経緯

  1. 2026-10-15NVIDIA团队在arXiv提交论文《Priming Base Models for Agentic Post-Training: An Open Midtraining Recipe》,公开中间训练的数据构造方法与消融实验结果。arxiv.org ↗
  2. 2026-10-17媒体广泛报道该配方,强调其在Nemotron系列模型上带来的SWE-bench等代码基准分数显著提升,以及其对Agent鲁棒性的改善。aiweekly.co ↗

未解決の問題

  • 该中间训练配方在非Nemotron架构(如Qwen、GLM)上的迁移效果如何?
  • 自然软件工程数据与少量Agent轨迹的最佳混合比例在不同规模模型中是否一致?
  • 社区复现该配方时,所需的计算资源门槛是否会限制中小实验室的应用?
トピック全文を読む →
リサーチを続ける →