Confident but Incorrect: Measuring LLM Confidence Calibration for Mathematical Problem-Solving

Large Language Models are increasingly being used as tools for mathematical problem-solving and have shown progressive improvement in reasoning capability. However, even frontier LLMs remain prone to hallucination when challenged with problems beyond their capabilities, often presenting incorrect reasoning with high confidence. In this paper, we examine how LLM self-assessed confidence aligns with empirical accuracy across math problems of varying difficulty and type. Furthermore, we test a model’s confidence on both its initial attempt and a prompted retry in which the model is informed of its prior error. Our method reveals a systematic overestimation of confidence that persists even after the model is informed of prior error, with miscalibration increasing largely monotonically with problem difficulty. These findings highlight the gap between perceived and actual LLM reliability in mathematical reasoning.