Practical guidelines for LASSO and competitors under dependence and scale effects

Simulation study assessing the impact of standardization on covariate selection and prediction for LASSO and its competitors in high-dimensional linear regression under dependence and scale effects. In a multiple linear regression model with p>1 covariates, the implementation of penalization techniques often implies a preliminary univariate standardization step. Although this prevents scale effects on the covariate selection procedure, possible dependence structures can be disrupted, leading to wrong results. This is particularly challenging in high-dimensional settings where p≥n. In this paper, we analyze the impact of standardization on the LASSO for different dependence and scale structures by means of an extensive simulation study. Two distinct objectives are pursued: accurate covariate selection and predictive performance. Additionally, LASSO's behaviour is compared with that of some well-known or innovative competitors. This comparison is also extended to three real datasets exhibiting these characteristics. Eventually, we conclude with a discussion and simulation-based practical guidelines for the simulation scenarios considered in this work, indicating the most suitable methodology in terms of covariate selection and prediction.