Calculated Formula Questions (CFQs) are a prevalent and critical assignment type in engineering courses to help students practice solving real-world problems. However, providing timely and personalized feedback on CFQ assignments remains challenging in large classrooms. Recent development of artificial intelligence (AI) offers unprecedented opportunities to deliver timely feedback through empowering intelligent tutoring systems (ITSs). Nevertheless, existing efforts have been constrained to assignments with readily available structured digital data, creating a gap in supporting unstructured CFQs. This study introduces a life-cycle framework that enables the development of an AI-powered ITS for CFQs, from data curation and model training to student-facing system deployment. Using graded CFQ assignments from undergraduate Engineering Economics courses as a case study, we built a digitalized dataset and applied a novel random masking technique to augment small-scale and imbalanced data. For each CFQ, we trained an eXtreme Gradient Boosting (XGBoost) model as its AI backbone. The model functions to predict potential mistakes in the solution using only each student’s submitted numerical answers, bypassing access to full written solutions. Our experiments demonstrate the feasibility of AI models in solution diagnosis, achieving an average precision of 0.81, a recall of 0.79, and an accuracy of 0.65 in predicting mistakes. To balance feedback efficiency and accuracy, we implemented a dialogue-based interaction scheme within a student-facing web interface. This scheme adaptively gathers additional inputs from students when the AI model’s predictions have close probabilities. Together, the AI backbone models and the web interface form an AI-powered ITS ( Arthur ) that delivers real-time and personalized feedback. Our framework offers a scalable pathway for building AI-powered ITS across engineering courses.
Yin et al. (Sun,) studied this question.