Answers are grounded in this paper and cite their sources.
Learning management system (LMS) data are widely used to identify students whomay require academic support. However, models developed from historical cohortsmay encounter behavioural change in later academic years. This study examineswhether early warning policies remain reliable under temporal change. LMS datafrom Accountancy and Global Economics courses across three academic years wereanalysed using observation windows from Week 4 to Week 12. A logistic regressionmodel based on activity intensity and active day regularity was evaluated usingchronologically ordered cohorts. The analysis considered discrimination,probability calibration, and the portability of decision thresholds selected tomatch support capacity. Discrimination remained useful in later cohorts, whilecalibration deteriorated in a cohort-dependent manner. Historical thresholds didnot preserve their intended workload when applied to later cohorts. At Week 12,a threshold targeting 20 percent of students selected 4.0 percent in Accountancy,but 89.4 percent and 69.0 percent in the two later Global Economics cohorts.Fixed-capacity ranking kept the number of selected students close to the intendedsupport capacity and continued to identify groups with a higher concentration ofrisk of not passing on the first attempt than random selection. These resultsdistinguish model portability from policy portability. Temporal evaluation ofearly warning systems should therefore consider both predictive performance andthe reliability of the decision rule used to allocate student support.
No takes yet. Share an insight, caveat, or question.
Babar et al. (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: