OpenAI Claims GPT-6.1 Sol Achieves 86.6% Accuracy in COBOL Migration Tests, Far Surpassing Previous Generation
OpenAI released new benchmark results showing significant improvements in handling low-resource programming languages like COBOL, aiming to aid legacy system modernization while noting that expert review remains necessary for production deployment.
ImportanceMaterialEvidenceE2 unreplicatedWrite-upQuick
On October 19, 2026, U.S.-based AI company OpenAI published research results detailing the performance of its new models on low-resource programming languages such as COBOL and Fortran. According to the company's official blog, the GPT-6.1 Sol model achieved an 86.6% accuracy rate in COBOL program migration tests under low reasoning intensity settings. This represents a substantial leap from the previous generation, GPT-5.6 Sol, which scored only 5.4% under identical conditions.
The tests focused on migrating COBOL programs to modern languages like C++, Java, or Python while preserving original behavior. OpenAI emphasized that successful migration requires byte-for-byte consistent output, checked via hidden tests. For instance, COBOL's fixed-point arithmetic truncates decimal places unless explicitly rounded; any deviation in the rewritten code would result in different financial outcomes despite running without errors.
OpenAI cited IBM data stating that over 40% of online banking systems rely on COBOL, supporting a vast number of card transactions and ATM operations. These advancements aim to help modernize critical infrastructure supported by older systems. However, the company noted that deploying these capabilities in production software still requires expert review and testing against specific business rules and dependencies.