An overview of variance-based importance measures in the linear regression context: comparative analyses and numerical tests

One of the most fundamental issues in many socio-environmental studies is the identification of causal effects and influential variables related to phenomena of interest. In the context of regression analysis, importance measures are effective tools for feature selection and model interpretation, allowing for the ranking of the most influential regressors. In particular, variance-based importance measures (VIMs) are a prominent topic in the field of statistics, as well as in the emerging field of global sensitivity analysis. This is due to their accessible interpretation as variance shares of the explained variable. This work focuses on the linear regression model and aims to provide an updated overview of the most well-founded methods, mainly from comparative analyses and numerical tests on various toy cases. The paper also addresses some of the practical challenges that arise, including the case of dependent inputs and high input dimensionality. The practical relevance of these tools is demonstrated through empirical studies on simulated data and public datasets. The Supplementary Material C also presents the use of VIMs in a classification context, specifically via the logistic linear regression model.