Assessing Learning Progression in a Vehicle Dynamics Course Using a Rubric-Based, GenAI-Assisted Open-Ended Approach

This study introduces a formative assessment approach for examining changes in cognitive demand across student responses using open-ended questions and rubric-based evaluation. The method is designed for small cohorts and emphasizes instructional refinement across predefined cognitive levels. It integrates Generative AI to support rubric application, scoring consistency, and identification of response patterns without replacing instructor judgment or established assessment practices. The approach addresses challenges in specialized programs with low enrollment, where large-scale statistical designs may be impractical, particularly in rapidly evolving fields such as automotive engineering. The methodology was applied in a vehicle dynamics course with 19 students, self-assigned to 6 teams of similar size by selecting an available slot; teams were then allocated to one of two learning paths by team number, in which the same activities were delivered in reversed order in a crossover design combining simulation-based and experiential activities. Assessments were aligned with Bloom’s taxonomy and ABET outcomes. Four Generative AI models (GPT-4 Turbo, GPT o4-mini, DeepSeek, and GPT-4o-mini) were used to support scoring under rubric and prompting procedures defined by an experienced instructor. Overall, inter-model agreement was moderate (Krippendorff’s α=0.666). This value did not meet the reference criterion of 0.7 adopted in the study and indicates meaningful model-dependent variation in rubric application. Within this limitation, task-level performance indicators showed a shift toward higher cognitive demand, particularly at the application level, with responses moving from conceptual definition toward early analysis. The approach also surfaced learning gaps that informed instructional adjustments, supporting its potential as a diagnostic tool for continuous improvement in engineering education.