Enhancing variable selection in elastic-net regression for high-dimensional data sets: The use of modified Bayesian type criteria and hybridized smoothed covariance estimators

Many academic subjects now require high-dimensional data analysis. You can choose the variables most crucial for accurate models in many ways. Choosing variables in high-dimensional datasets is difficult due to their complex relationships, especially when considering main and interaction effects. Because of this complexity, applying conventional criteria like AIC and BIC is inappropriate; instead, Bayesian-type information criteria like EBIC, mBIC, and mBIC2 should be used. The elastic net model was employed in this study, and its hyperparameters were chosen using these criteria (EBIC, mBIC, and mBIC2). Lambda was determined using smoothed covariance estimators. We investigated the effect of the penalty term (c-value) on these criteria and how it influences their performance. Numerous Monte Carlo simulations with varying correlation structures, variable counts, and sample sizes were conducted, and we also applied the method to a real dataset. Based on the findings, smaller c-values lead to better predictive accuracy and fewer selected variables. As the c-value increases, the number of variables chosen rises, typically resulting in higher MSE. Using stabilized or smoothed covariance estimators provides better results in high-dimensional data. Based on the lowest MSE values, EBIC and mBIC perform better with high-dimensional data and higher correlations.