Performance Evaluation of Multinomial Logistic Regression, Random Forest, and XGBoost Methods in Data Classification

The development of data volume and complexity in the digital era increases the need for effective classification methods to support decision-making. Decision-making in classification tasks often requires methods that are well-suited to the data, along with the ability to produce accurate and reliable predictions. As scientific knowledge continues to advance, a wide range of classification methods have been developed. This study aims to analyze the performance of three commonly used classification methods Multinomial Logistic Regression, Random Forest, and XGBoost, in handling diverse data characteristics. Ten varied public datasets were used in this research, with differences in the number of classes, features, instances, balanced and imbalanced data conditions. Evaluation was conducted based on accuracy, F1-score, precision, and recall. The analysis results show that Random Forest consistently delivers the best performance particularly on imbalanced data. XGBoost demonstrates superiority on more complex datasets, while Multinomial Logistic Regression proves more effective on relatively small datasets. This research provides valuable insights into selecting appropriate classification methods based on data characteristics and highlights the effectiveness of ensemble-based approaches in handling diverse data. Based on the findings, it is recommended that the selection of classification algorithms be tailored to the characteristics of the dataset. Random Forest is preferable for handling imbalanced data, while XGBoost is ideal for complex datasets requiring robust hyperparameter tuning. Multinomial Logistic Regression remains a viable option for simpler datasets with fewer observations and features. Future research could explore hybrid models that combine these approaches to further optimize classification performance across various domains.