OpenAI Says New Models Achieve 86.6% Accuracy in COBOL Migration Tests

OpenAI announced on October 19, 2026, that its latest models have achieved substantial breakthroughs in coding capabilities for low-resource programming languages such as COBOL, Fortran, and Visual Basic. These languages, while having less available training data online, support critical global infrastructure, including over 40% of online banking systems, 80% of credit card transactions, weather forecasting, and nuclear safety analysis.

To measure this progress, OpenAI developed two specialized evaluations. The first, MultiLangBench, requires models to build a SQLite-compatible database engine from scratch across 20 different programming languages. Data shows that GPT-6 Astra achieved an average test pass rate of 87.4%, a significant increase from 26.8% for GPT-5.6 Sol. Specifically, the pass rate for COBOL jumped from 10.9% to 72.1%, and Fortran improved from 21.4% to 83.5%. The lower-cost model GPT-6 Luna also reached an average pass rate of 40.3%, with API costs approximately 78% lower than the previous generation.

The second evaluation focuses on COBOL code migration, testing whether models can convert existing COBOL programs into modern languages like C++, Java, or Python while strictly preserving original business logic and behavior. Since COBOL is often used for financial transactions, specific arithmetic rules (such as default truncation rather than rounding) are crucial. OpenAI reports that GPT-6.1 Sol achieved 86.6% migration accuracy at low reasoning effort, compared to just 5.4% for GPT-5.6 Sol; at higher reasoning efforts, accuracy approached saturation at 97.9%.

OpenAI stated that these improvements aim to help teams maintain and modernize software dependent on legacy systems, addressing challenges like incomplete documentation and developer turnover. The company has open-sourced the MultiLangBench evaluation setup and test suite on GitHub for developers to verify performance gaps in specific languages. Despite the model's enhanced performance, OpenAI emphasizes that applying such automated migration techniques in production environments still requires expert review and testing against business rules.