Leakage-Aware, Activity-Controlled Evaluation Protocol for Academic-Risk Prediction

Early identification of at-risk students supports timely intervention, and graph-based models are increasingly applied to learning-analytics data for this purpose. However, reported gains can be inflated by two issues at the evaluation stage: temporal leakage, when information from after the prediction cutoff enters features or splits; and activity-level confounding, when structural signals largely restate an underlying level of student activity. This paper does not propose a new forecasting model; it proposes a leakage-aware, activity-controlled auditing protocol that integrates cutoff-respecting splits, a future-information audit, and a conditional incremental test into a reproducible procedure for testing whether graph-derived structural signals provide incremental information beyond activity-level covariates. We apply the protocol across four OULAD course presentations spanning two academic years, two start seasons, and two subject areas. Across all presentations, admitting post-cutoff information inflates F1 by roughly 0.15 to 0.30, an effect comparable to or larger than the differences among the evaluated models. Once activity covariates are controlled, connectivity provides limited incremental value: AUC changes remain within a few thousandths, and a propagation-based index is nearly rank-equivalent to node degree (Spearman rho approximately 0.996 to 0.999). Across three presentations, detected communities align closely with course modules (NMI approximately 0.99; purity 1.0); because co-access edges are induced by module-specific resources, this alignment is best read as a diagnostic property of the graph construction rather than as evidence of independent risk groups. These results are consistent across presentations, indicating that the protocol's conclusions are not driven by a single cohort. This is a preprint and has not been peer reviewed.