An Optimized Feature Selection Approach for Enhanced Credit Scoring Prediction: A Hybrid Rf-L1 and Logistic Regression Model

Feature selection is a critical step in machine learning and data analysis to enhance model performance by identifying the most relevant features while reducing dimensionality. This study proposes RF-L1(Recursive Feature elimination-lasso L1), a hybrid feature selection method that integrates Recursive Feature Elimination (RFE) and Lasso-L1 Regularization to optimize feature selection and improve classification performance. Three approaches are compared: RFE-based selection, Lasso-L1 Regularization and the proposed RF-L1 method. RFE iteratively removes less important features, while Lasso-L1 Regularization selects features by applying penalization. The RF-L1 approach combines both techniques, using RFE for initial selection and Lasso-L1 for refinement. Logistic regression is applied for classification, with performance examined utilizing F1-score, recall, accuracy, precision, and area under the ROC curve. The optimal feature set is identified by comparing the weighted sum of selected features and evaluation metrics across all methods. Results demonstrate that the proposed RF-L1 based Logistic Regression enhances model efficiency and robustness, demonstrating its potential to outperform traditional feature selection techniques.