Does ChatGPT influence metacognitive calibration during undergraduate physics problem solving?

We present a controlled pilot study examining how access to generative Artificial Intelligence influences metacognitive calibration, the alignment between perceived and actual learning. Second-year undergraduate physics students of the University of Bath were split into a control group, completing three successive first-year exam problems of comparable difficulty under standard exam conditions, and an experimental group, in which ChatGPT access was permitted during the intermediate problem. We quantify the performance improvement of students via a partial-credit marking scheme, while their perceived improvement is recorded through self-assessment questions. Students with access to ChatGPT show a ∼ 60% improvement in calibration accuracy, primarily associated with a reduction in underconfidence. Our exploratory study suggests that the educational value of generative AI may extend beyond performance by supporting students in forming more accurate assessments of their own performance, and in turn, understanding.