Improving Source Code with Assistance from AI — A Pilot Case Study with ChatGPT

ChatGPT queries were used to provide feedback on five C++ programs selected from various programming assignments for two graduate-level computer science courses – a scientific programming course and an algorithms course. The evaluated software was written by the first author for those courses within the last two years. ChatGPT was asked to evaluate and provide feedback for each program. Specifically, ChatGPT was asked to evaluate the code for strengths and weaknesses and make recommendations for improving (1) execution speed as well as (2) readability and maintainability. A subjective agreement rating was generated by the authors for each strength, weakness, and recommended change provided by ChatGPT. While the overall agreement with the ChatGPT provided feedback was over 90 percent, at times, ChatGPT's recommendations were found misleading.