Data-driven decision making in school leadership: AI-based academic performance prediction using ML and SDT motivation

Purpose This exploratory study aims to develop and evaluate artificial intelligence (AI) based predictive models for academic outcomes, offering data-driven insights for school leadership by integrating machine learning (ML) with self-determination theory motivation frameworks. Design/methodology/approach A comparative analysis was conducted across eight regression models, including deep neural network (DNN), random forest (RF) and gradient boosting (GB), using a small-scale dataset (n = 68). Model performance was assessed through a robust 5× repeated 5-fold cross-validation (CV), utilizing root mean squared error (RMSE) as the primary metric. The statistical significance of the models was validated via 100 permutation tests. To provide actionable transparency, the best-performing model was subjected to Shapley additive explanations (SHAP) analysis and learning curve (LC) analysis to evaluate generalization capabilities and bias-variance tradeoffs. Findings The RF model emerged as the superior performer, achieving the lowest Mean RMSE of 5.2138 (±1.5280), followed by KNearest Neighbors (5.3572) and light GB machine (5.5326). Statistical testing confirmed significant predictive power for RF (p = 0.0198) and GB (p = 0.0297). SHAP analysis identified first-quarter scores, Age and study time as primary predictors, but critically highlighted SDT-related factors such as autonomy importance, relatedness and recommendation likelihood (a proxy for AI app engagement) as top-10 influential features. The LC indicated a persistent generalization gap, suggesting that while the model captures complex patterns, its current predictive stability is constrained by the small sample size (n = 68). Research limitations/implications The primary limitation is the small sample size (n = 68). While CV and permutation tests were used to ensure model stability, the small N means that the results are highly sensitive to the specific characteristics of this cohort. Consequently, these findings cannot be generalized to broader or more diverse student populations without further large-scale validation. High variance and overfitting as evidenced by the LC, a persistent gap remains between the training and CV RMSE. This indicates a high degree of variance, where the model is still prone to overfitting the noise within the small dataset. The model's predictive accuracy might fluctuate significantly if applied to a different academic environment. Exploratory Nature of Policy Claims. The practical implications discussed such as using specific SHAP features to drive curriculum changes should be viewed as hypotheses for future research rather than definitive institutional mandates. The synthetic nature of some data labels and the limited demographic range further necessitate a cautious approach to applying these results to broad educational policy. Constraints of deep learning: The relative underperformance of the DNN (MLPRegressor) further highlights the difficulty of applying complex, “data-hungry” architectures to small-scale educational datasets. Practical implications While the RF model and SHAP analysis provide valuable insights, it is important to treat these findings as exploratory and preliminary due to the study's specific context and sample size. Nevertheless, this investigation offers a promising framework for how school leadership can move toward a more proactive, data-informed ecosystem. Targeted interventions and resource allocation: The identification of “score first quarter” and “study time” as primary drivers suggests that academic support should be front-loaded. Rather than waiting for mid-year failures, school leaders can use early quarter data as a “screening tool” to identify students who may require additional mentorship. However, given the exploratory nature of this study, such interventions should be implemented as pilot programs to further validate these predictors in real-world settings. Originality/value This study uniquely integrates statistical significance testing with comprehensive explainable AI to bridge the gap between ML and educational psychology. By identifying Autonomy and Personalization as measurable predictors of academic success, it provides a methodological proof-of-concept for how school leadership can leverage SDT-informed AI. Given the exploratory nature and limited sample, these results serve as a preliminary foundation for proactive, motivation-aware educational strategies and targeted digital interventions.