Which Imputation Fits Which Feature Selection Method? A Survey-Based Simulation Study

Tree-based learning methods, such as Random Forest and XGBoost, remain the gold standard for prediction in tabular data. Feature importance measures are frequently employed in feature selection and to evaluate the influence of features on the outcome variables within the model. This approach extends to survey data, which are prevalent in social sciences and official statistics, presenting a significant challenge related to missing values. A typical strategy is to impute missing data before applying the learning method. However, with the sheer number of possible imputation techniques, determining the optimal method to best reflect feature importance and feature selection in subsequent analyses is a complex question. In this paper, we investigate this issue using a survey-based simulation study for eight state-of-the-art imputation methods and three tree-based learners. The imputation methods encompass listwise deletion, three MICE options, four missRanger options, and the recently proposed mixGBoost imputation approach. As learners, we evaluate the performance of the two most widely used tree-based methods–Random Forest and XGBoost–alongside an interpretable, regularized linear model (Elastic Net).