Pass or Fail if One Byte is Different: What is the Accuracy of OpenAI's New Model for COBOL Migration?
OpenAI evaluated its new model's programming capabilities, reporting that the average pass rate for building database engines in 20 languages increased from 26.8% to 87.4% compared to previous models.
Key points
- •The evaluation tested how well AI can handle COBOL and Fortran, which have limited training resources.
- •In a test building database engines in 20 languages, the new model's average pass rate rose from 26.8% for previous models to 87.4%.
- •The article also discusses the results of evaluating the rewriting of existing COBOL programs into other languages.