독서리서치레이더투자 프레임워크
로그인 / 회원가입
← 읽기로 돌아가기
뉴스주제 · GPT-6.1 Sol중대2 분 소요DisconfirmAI 편집부

OpenAI Says New Models Achieve 86.6% Accuracy in COBOL Migration Tests

OpenAI introduces MultiLangBench and COBOL migration evaluations, showing significant improvements in code generation and behavior preservation for legacy systems.

2026-10-19 20:00 GMT+8

OpenAI announced on October 19, 2026, that its latest models have achieved substantial breakthroughs in coding capabilities for low-resource programming languages such as COBOL, Fortran, and Visual Basic. These languages, while having less available training data online, support critical global infrastructure, including over 40% of online banking systems, 80% of credit card transactions, weather forecasting, and nuclear safety analysis.

To measure this progress, OpenAI developed two specialized evaluations. The first, MultiLangBench, requires models to build a SQLite-compatible database engine from scratch across 20 different programming languages. Data shows that GPT-6 Astra achieved an average test pass rate of 87.4%, a significant increase from 26.8% for GPT-5.6 Sol. Specifically, the pass rate for COBOL jumped from 10.9% to 72.1%, and Fortran improved from 21.4% to 83.5%. The lower-cost model GPT-6 Luna also reached an average pass rate of 40.3%, with API costs approximately 78% lower than the previous generation.

The second evaluation focuses on COBOL code migration, testing whether models can convert existing COBOL programs into modern languages like C++, Java, or Python while strictly preserving original business logic and behavior. Since COBOL is often used for financial transactions, specific arithmetic rules (such as default truncation rather than rounding) are crucial. OpenAI reports that GPT-6.1 Sol achieved 86.6% migration accuracy at low reasoning effort, compared to just 5.4% for GPT-5.6 Sol; at higher reasoning efforts, accuracy approached saturation at 97.9%.

OpenAI stated that these improvements aim to help teams maintain and modernize software dependent on legacy systems, addressing challenges like incomplete documentation and developer turnover. The company has open-sourced the MultiLangBench evaluation setup and test suite on GitHub for developers to verify performance gaps in specific languages. Despite the model's enhanced performance, OpenAI emphasizes that applying such automated migration techniques in production environments still requires expert review and testing against business rules.

로그인 / 회원가입
독서리서치레이더투자 프레임워크

출처

  • openai.com↗
주제

GPT-6.1 Sol

OpenAI推出的GPT-6.1 Sol模型在低资源编程语言处理上取得显著突破,特别是在银行核心系统常用的COBOL语言迁移测试中,准确率达到86.6%,远超上一代模型的5.4%。这一进展旨在解决全球关键基础设施中老旧代码维护难、文档缺失及…

3 단계1 건업데이트됨 2026-10-20 21:51

현재까지의 과정

  1. 2026-09-30OpenAI发布GPT-6.1 Sol,定价为旗舰版Astra的五分之一,主打高性价比的专业工作负载。openai.com ↗
  2. 2026-10-01LessWrong用户Rauno Arike发布实测报告,显示GPT-6.1 Sol在无思维链模式下追平Astra差距的80%,提出循环架构假说。lesswrong.com ↗
  3. 2026-10-19OpenAI公布MultiLangBench和COBOL迁移评估结果,称GPT-6.1 Sol在低推理强度下COBOL迁移准确率达86.6%。openai.com ↗

아직 해결되지 않은 문제

  • GPT-6.1 Sol是否确实采用了与Astra相同的循环Transformer架构?
  • 在缺乏完整文档的情况下,AI辅助的COBOL迁移在实际金融系统中的错误率如何控制?
  • OpenAI是否会进一步公开更多低资源语言的基准测试细节以供外部复现?

이 스레드의 다른 내용

  • GPT-6.1 Sol's no-CoT runs close 80% of the gap to Astra2026-10-01
전체 주제 읽기 →
리서치 계속하기 →