Evaluating ChatGPT’s Engineering-Reasoning Capabilities and Constraints Through Examples from Mechanical-Engineering Education

This paper evaluates the mechanical engineering reasoning capabilities of ChatGPT-4, an advanced Large Language Model (LLM), with the aim of enhancing mechanical engineering education. Mechanical engineering education extends beyond text comprehension, a domain for which ChatGPT is renowned. It aims to nurture future engineers to become critical thinkers, who are proficient in applying acquired knowledge to execute complicated engineering tasks. However, there's an apparent gap in understanding how ChatGPT can be effectively integrated into educational practices within this specialized area due to a lack of detailed insights into its abilities and limitations. This research seeks to fill this void by exploring and assessing ChatGPT's reasoning abilities and limitations within the context of mechanical engineering. It examines the capabilities and constraints of ChatGPT in engineering reasoning by analyzing two mechanical examples, which are drawn from machine design and dynamics. By comparing ChatGPT's entire reasoning process and individual steps with human reasoning, this investigation unveils both its constraints and capacities. The results show that ChatGPT's limited capability to understand the profound implications of text. It addresses the need for caution when employing it in reasoning tasks within the context of mechanical engineering education.