Comparative Analysis of Gradient Boosting, XGBoost, and KNN on Predicting Student Graduation in Imbalance and Balance Data Schemes

The objective of this research is to compare the performance of three machine learning algorithms: Gradient Boosting, XGBoost, and K-Nearest Neighbors (KNN) in predicting student graduation using a quantitative approach and comparative experimental methods. The analysis process follows the CRISP-DM stages, which include business understanding, data understanding, data preparation, modeling, evaluation, and implementation. The dataset used consists of approximately 1,251 data points from students of the 2018–2020 cohort with an imbalanced distribution, namely 73.78% graduated on time and 6.22% did not graduate on time. The variables analyzed include academic and non-academic data, such as total credits, GPA per semester, number of repeated courses, and number of leaves. To address the data imbalance, the SMOTE-TOMEK balancing technique was applied. The results of this research indicate that XGBoost showed an improvement in performance after balancing, with accuracy, precision, recall, and F1-score reaching 1.0000. Gradient Boosting shows consistent performance with a score of 0.9992, both before and after balancing. KNN also experienced an increase in accuracy from 0.9928 to 0.9968 after the balancing process. Findings from the confusion matrix results show a significant improvement in classification. Therefore, the implementation of the SMOTE-TOMEK technique has proven effective in improving the performance of classification models on imbalanced data, and XGBoost is recommended as the main algorithm for predicting student graduation.