A Scalable and Interpretable Framework for Early Identification of Recommendation-Oriented Learner Categories

The Psychological Assessment master’s course has undergone a substantial redesign since 2020, introducing multiple formative assessments to support active and self-regulated learning. This digitally enriched course environment enables the collection of fine-grained behavioral and performance data throughout the semester. The primary objective of this study is to develop and evaluate a robust and scalable framework that supports the provision of personalized learning recommendations using assessment data available in real time once the relevant formative assessments have been completed. Our approach uses separate principal component analyses to aggregate time-investment and performance variables into two interpretable dimensions. Median-based segmentation of the resulting component scores defines four recommendation-oriented learner categories. Although intentionally simple and transparent, the framework adapts dynamically as assessment data accumulate over the semester. Bootstrap analyses of the principal components and cross-cohort transferability tests demonstrate stable component structures and consistent prediction trajectories across three cohorts, including adjacent-year and two-year transfer scenarios. Predictive performance improves steadily as assessment data accumulate and stabilizes after the third formative assessment, administered around week 9, after which additional gains become progressively smaller. Remaining misclassifications are primarily concentrated near the median decision boundaries, consistent with the continuous distributions of time investment and performance. This enables timely mid-semester identification of recommendation-oriented categories that can inform targeted pedagogical goals related to motivation, learning strategies, and appropriate levels of challenge. Overall, the findings show that recommendation-oriented learner categories can be derived from routinely collected formative-assessment data, predicted early enough to support pedagogical intervention, and transferred consistently across cohorts. The proposed framework therefore provides a structurally robust and interpretable foundation for scalable personalized support in courses with recurring formative assessments.