Benchmarking AI Mathematical Capabilities: A Comprehensive Evaluation of LLMs in Solving Integration Problems with Error Correction Analysis

This study comprehensively evaluated seven leading Large Language Models (Chat GPT 4o, Gemini Advanced with 1.5 Pro, Copilot Pro, Claude 3.5 Sonnet, Meta AI, Mistral AI, and Perplexity) on Calculus II integration problems through a progressive testing framework consisting of three incremental tests and a final examination conducted between September and December 2024. The evaluation covered various integration techniques, including substitution, integration by parts, partial fractions, area and volume calculations, and series convergence tests, totaling 420 possible points. A key methodological innovation was the implementation of structured follow-up prompting to assess models' error correction capabilities. Results showed significant performance variations, with Gemini Advanced with 1.5 Pro achieving the highest score of 91.7% (A-), followed by Chat GPT 4o with 89.3% (B+), and an overall class average of 83.3% (B). While all models demonstrated strong proficiency in basic integration techniques (averaging 95.2%), they struggled significantly with geometric applications requiring spatial reasoning, with failure rates exceeding 70% on volume of revolution problems. The follow-up analysis revealed that computational errors were readily correctable (90%+ success rate), whereas conceptual misunderstandings, particularly in geometric visualization, persisted despite explicit guidance. These findings demonstrate that while current LLMs have achieved undergraduate-level competency in procedural mathematics, they face fundamental limitations in spatial-geometric reasoning and conceptual understanding, suggesting they can serve as valuable computational tools but cannot replace human instruction for developing mathematical intuition.