Family of shrinkage estimators in binary logistic regression for data classification

Computer-based classification methods have become increasingly popular as data structures become more complex. Binary logistic regression is a popular classification method, and to handle complex data structures with logistic regression, many regularization methods have been proposed using penalty terms instead of classical estimation methods. This paper shows that regularization methods previously proposed for logistic regression estimation can also be used for classification. To do so, we will use iterative ridge, iterative Liu, and iterative two-parameter estimators. In addition, we compare these estimators with random forest, support vector machine, and XGBoost on six different data sets. The experimental results show that the optimal classifier depends on the characteristics of the underlying data. The iterative ridge estimator performed best under strong multicollinearity and on probability-based metrics, while the random forest provided superior discrimination performance in datasets with weaker multicollinearity and larger sample sizes. The findings also suggest that iteratively biased logistic estimators are interpretable and competitive classification alternatives where multicollinearity and coefficient stability are important, while random forest, support vector machine and XGBoost may be advantageous in cases where nonlinear or interaction-oriented estimation patterns are dominant or in imbalanced data situations.