Evaluation of accuracy and potential biases of imputation methods on biomedical datasets subject to different mechanisms and patterns of missingness

Objective : Accurate and unbiased statistical and machine learning (ML) analyses require data to be complete. Yet biomedical datasets are usually incomplete due to data missing at random (MAR), missing not at random (MNAR), or missing completely at random (MCAR). While imputation is commonly performed, analyses can be biased based on the type of missingness and the pattern of missing entries, including structured missingness (SM). To advance the goal of accurate clinical data analysis, this study employs a rigorous methodology to evaluate accuracy and potential biases of imputation methods on biomedical datasets subject to different mechanisms and patterns of missingness. Methods : De-identified biomedical data were obtained through retrospective chart review of COVID-19 patients at our institution’s Hospital. Clinically relevant imputation techniques were explored with data at various percentages and patterns of missingness. Single (SI) and multiple imputation (MI) performances were evaluated using a variety of metrics, including raw bias of mean, standard deviation, coverage, normalized root mean square error (NRMSE), and negative logged Wasserstein’s distance (NLWD) metric. To measure imputation performance, classification models were trained on the imputed datasets. Results : MI methods better reconstructed data compared to SI methods as measured by NLWD. While SI methods underestimated standard deviation, MI accurately estimated standard deviation except for MNAR. Both MI and SI methods achieved similar performance for bias of the mean and coverage, while SI methods achieved lower NRMSE. Conclusion : This study offers a rigorous methodology to evaluate imputation performance on biomedical datasets subject to different mechanisms and patterns of missingness.