Comparative Analysis of Machine Learning Algorithms for Software Fault Prediction

Software fault prediction helps development teams prioritize quality assurance by identifying software components that are likely to contain faults before release. The purpose of this study is to determine whether three recommended optimization strategies: parameter optimization, feature selection, and class-imbalance handling produce consistent performance improvements across different machine learning algorithms. This study compares five machine learning algorithms for software fault prediction, namely Random Forest, K-Nearest Neighbor, AdaBoost, Support Vector Machine (SVM), and a fully connected neural network, using historical change features derived from five open-source software projects. The study evaluates three optimization strategies: parameter optimization, feature selection using Least Absolute Shrinkage and Selection Operator (LASSO) regression, and class-imbalance handling through oversampling and undersampling. Model performance is assessed using five-fold stratified cross-validation with Accuracy, Receiver Operating Characteristic–Area Under the Curve (ROC-AUC), and F1-score. The results show that the parameter-optimized baseline models achieved comparable performance across algorithms, with accuracy values above 0.83 and ROC-AUC values modestly above random performance. Feature selection produced limited overall gains and improved all three performance measures only for AdaBoost, while class-imbalance handling generally reduced accuracy although it improved ROC-AUC in several cases.