Performance of ChatGPT in solving wave problems

The ability of ChatGPT, especially its latest versions, attracts significant attention from educators and researchers in fields such as physics. Several studies on this topic explore the performance of ChatGPT in answering physics questions across different versions of ChatGPT. In this study, we evaluate the development of ChatGPT in answering physics questions using the Mechanical Waves Conceptual Survey (MWCS) to examine whether its capabilities improve across different versions. A total of 75 responses per item are generated using ChatGPT versions 4, 4.o, and 5.2, resulting in 4950 responses for comprehensive analysis. Based on the comparison between the three versions of ChatGPT, we find that some difficult types of questions that are answered incorrectly by previous versions, as reported in earlier studies, can now be answered correctly, although there are still minor mistakes, especially when examining the reasoning. A generalized linear mixed model (GLMM) analysis, together with comparisons, indicates that ChatGPT 5.2 significantly outperforms both ChatGPT 4 and 4.o, suggesting that the capabilities of the latest version included in this study significantly improve. These findings highlight a future direction for this type of research, as our study shows that the latest version of ChatGPT can correctly answer more types of questions in MWCS.