Bridging algorithmic prediction and teacher judgment: an explainable AI framework with institutional fairness calibration for early warning systems

Prediction models for identifying at-risk students have become technically mature, yet their integration into secondary school practice remains limited by two persistent obstacles: opacity in model outputs, which erodes teacher trust, and uneven predictive performance across institutions, which risks widening rather than closing educational equity gaps. This study develops and evaluates an integrated five-stage framework combining systematic ablation analysis, SHapley Additive exPlanations (SHAP)-based decomposition, and school-level isotonic regression calibration for educational early warning systems. The framework is empirically evaluated on physics assessment data from 497 Grade 11 students across four Indonesian secondary schools (SPHERE dataset; intraclass correlation coefficient ICC = 0.309) and cross-validated on the UCI Student Performance cohort. Ablation results show that, within a school that already has local training data, models trained solely on observable student data achieve R 2 = 0.354 on held-out data, establishing viable backup screening capability when teacher assessments are absent; incorporating teacher prediction raises performance to R 2 = 0.494 [ Δ R 2 = 0.140 ; paired test-set bootstrap 95% CI (0.069, 0.213), positive across 100 independent splits]. However, the teacher prediction variable correlates substantially with the target ( r pb = 0.626 ), so its SHAP dominance partly reflects outcome overlap rather than unique professional insight; a single-predictor model using only teacher prediction achieves R 2 = 0.275 , confirming that the remaining features contribute additional signal. SHAP analysis reveals a stable three-tier feature hierarchy—teacher-encoded signal, school context, and student attitudes and skills—with Jaccard top-5 rank stability exceeding 0.70 across three datasets. Leave-one-school-out cross-validation ( R 2 = − 0.281 ) demonstrates that the model cannot generalize to entirely unseen schools, while within-school models achieve R 2 ≈ 0.39 − 0.55 , confirming student-level predictive capacity when school context is held constant. Gender-disaggregated fairness analysis shows approximate equal opportunity across genders ( Δ TPR = 0.048, Δ FPR = 0.017). Isotonic regression calibration reduces institutional performance gaps when per-school training data is sufficient (School 4: Δ R 2 = + 0.082 ); a minimum of approximately 50 training students per school is identified as a practical deployment boundary condition. Initial cross-context validation across Indonesian and Portuguese datasets supports the framework’s methodological transferability, though the specific parameter estimates require multi-country replication before broader generalization can be claimed.