LeituraPesquisaRadarFramework de investimento
Entrar / Cadastrar
← Voltar para a leitura
NotíciasTópico · OpenAI多语编程基准Material1 min de leituraEditorial DisconfirmAI

OpenAI claims GPT-6 shows major accuracy gains in legacy programming languages like COBOL

OpenAI released new benchmark data claiming significant improvements in handling legacy code, though results are self-reported.

2026-10-21 00:40 GMT+8

OpenAI announced on October 20, 2026, that its latest AI models have shown significant progress in handling "low-resource" programming languages. These include COBOL, Fortran, and Visual Basic. While these languages have far less available training code than modern languages like Python, IBM estimates that COBOL alone powers more than 40 percent of online banking systems and 95 percent of all ATM transactions worldwide.

According to two evaluations built by OpenAI, the GPT-6 Astra model achieved an average test pass rate of 87.4 percent across 20 languages on MultiLangBench, a substantial increase from approximately 38 percent for the previous generation, GPT-5.6 Terra. The MultiLangBench task requires models to build a SQLite-compatible database engine from scratch.

On a specific COBOL migration benchmark, which involves translating COBOL into C++, Java, Kotlin, Python, or Rust, GPT-6.1 Sol reached 86.6 percent accuracy at low reasoning effort, compared to just 5.4 percent for GPT-5.6 Sol. Increasing the reasoning effort pushed the accuracy up to 97.9 percent.

It is important to note that these figures are based on OpenAI's internal evaluations and have not been independently reproduced. Furthermore, the high accuracy rates were achieved under specific configurations, such as maximum reasoning effort, which may not reflect typical production use cases due to cost and latency constraints.

Fontes

Entrar / Cadastrar
LeituraPesquisaRadarFramework de investimento
the-decoder.com↗
Tópico

OpenAI多语编程基准

OpenAI于2026年10月19日发布新基准测试MultiLangBench,专门评估大模型在COBOL、Fortran等“低资源”编程语言中从零构建SQLite兼容数据库引擎的能力,并公布GPT-6系列模型在遗留代码迁移任务上的显著性能…

3 etapas2 peçasAtualizado 2026-10-21 14:52

Como chegamos aqui

  1. 2026-10-19OpenAI在GitHub开源MultiLangBench基准测试套件,要求模型在禁止联网和移除现有SQLite库的环境下,依据文档用指定语言实现SQLite 3.40.1功能。github.com ↗
  2. 2026-10-19OpenAI发布官方博客《Advancing coding in low-resource programming languages》,披露GPT-6 Astra在MultiLangBench上平均通过率87.4%,以及GPT-6.1 Sol在COBOL迁移评估中高达97.9%的准确率。openai.com ↗
  3. 2026-10-20多家科技媒体(如iThome、The Decoder)报道OpenAI的新基准,指出COBOL仍是最难的语言(Astra通过率72.1%),并强调数据为厂商自测,未经独立验证。ithome.com.tw ↗

O que ainda está indefinido

  • 第三方机构或开发者社区能否在相同条件下复现OpenAI报告的GPT-6系列高通过率数据?
  • 在实际企业遗留系统迁移项目中,AI生成的代码是否真能保持业务逻辑的字节级一致性,还是仅通过了基准测试的特定用例?
  • OpenAI是否会公开其未发布的COBOL迁移评估的具体测试集和评分标准以供外部审计?

Também nesta thread

  • OpenAI Releases MultiLangBench to Evaluate AI Coding in Low-Resource Languages like COBOL2026-10-20
Ler o tópico completo →
Continuar pesquisando →