OpenAI Reports 97.9% Accuracy in Legacy Code Migration Benchmarks with New Models
OpenAI announced significant improvements in migrating COBOL, Fortran, and Visual Basic code, though the company notes that production use still requires expert review.
US artificial intelligence company OpenAI announced on October 19 that its latest generation models demonstrate significantly better performance in legacy programming languages such as COBOL, Fortran, and Visual Basic. These languages have less training data available but still support critical infrastructure in finance and government sectors globally.
According to TechNews, OpenAI designed two internal evaluations. The first, MultiLangBench, required models to build a SQLite-compatible database engine from scratch using 20 different programming languages. Results showed that GPT-6 Astra achieved an average pass rate of 87.4% under maximum reasoning settings, far exceeding the previous generation GPT-5.6 Sol's 26.8%. Specifically, COBOL performance improved from 10.9% to 72.1%, Fortran from 21.4% to 83.5%, and Visual Basic from 27.5% to 89.4%.
The second evaluation focused on code migration, requiring models to rewrite COBOL programs into C++, Java, Kotlin, Python, or Rust while maintaining original behavior. OpenAI stated that these tasks involve more than simple translation; they require preserving original calculation logic and business rules. Hidden tests were used to check for byte-for-byte consistency. Data indicated that GPT-6.1 Sol achieved 86.6% accuracy at low reasoning settings, up from 5.4% for GPT-5.6 Sol; with increased reasoning intensity, accuracy rose to 97.9%.
OpenAI mentioned that the lower-cost version, GPT-6 Luna, also achieved an average pass rate of 40.3% on MultiLangBench, with lower estimated API costs per test sample compared to the previous generation. However, the company emphasized that these results represent improvements in a test environment. For actual production use, expert review and careful verification against specific business rules and dependencies are still required.