Missing in action – where have all the data gone!?

Dear Ms Method Matters, I am currently preparing to conduct a retrospective study on the effect of postoperative haemoglobin concentrations after hip replacement surgery and total length of stay, the theory being that the total length of hospital stay is inversely correlated to haemoglobin concentrations. We have data on around 3000 cases of hip replacement on file, the only thing is that around 10% of the cases did not have postoperative haemoglobin concentrations measured. I did consider taking those patients with missing data out of the database, and so will analyse the dataset without them, but not only will this substantially reduce my case number, but I am also not sure whether deleting these cases would bias my dataset. I have read that there are methods to ‘impute’ missing data, but am not sure how this should be done, and how much missing data can I feasibly tolerate in my dataset? Filling in the Gaps Dear Filling in the Gaps, Almost all studies, regardless of whether they are prospective or retrospective, have some missing observations. Missing data can be broadly defined as ‘missing completely at random’ (3, 5 in Box 1), ‘missing at random’ (4 in Box 1) and ‘missing not at random’ (1, 2 in Box 1). There are broadly three main ways of handling missing data: omitting all the individuals who do not have a complete dataset (known as complete case analysis); omitting all the variables which have many missing values; and estimating, (imputing) the missing values. Before you decide how to handle missing data, you should think carefully about how the missing data can cause bias (please refer to Box 2). If the data are ‘missing not at random’, then it is better not to attempt any analysis which will generate misleading results. If the data are missing at random, then several methods could be used to estimate the missing values. Popular methods of data imputation include: replacing missing values with values imputed from the observed data (e.g. filling in all the gaps with the mean of the observed values); or filling in the missing value with the last known value for that patient. Neither of these two approaches are statistically valid, as these ‘single imputation’ methods cause standard errors in the group to be too small, and it is suggested that readers do not use these methods. In particular, imputing missing values with the last known value for that patient would distort the dataset because it is clinically obvious that there should be a difference between pre-surgical and post-surgical haemoglobin concentrations 1, regardless of whether the patient received a transfusion. Multiple imputation is another approach to handle the problem of missing data 2, and it is easily undertaken with common statistical software packages. With multiple imputation, multiple copies of the dataset are created, with the missing values replaced by imputed values which are sampled from their predictive distribution based on the observed data. Because we can never be sure what the missing values are, multiple imputation aims to allow for the uncertainty by creating several different plausible imputed data sets and combining results obtained from each of them. Multiple imputation has been used in several recent publications in Anaesthesia 3-5. Statistical software may use different methods to impute the missing values. For example, the fully conditional specification (as used in Shahin's and Klein's studies 3, 5) presumes that the missing data is arbitrary, whereas the monotone imputation method presumes that if a study follows a subject for 5 years, and a particular subject withdraws from the study during year three, missing data will occur during the subsequent years. Some software can analyse patterns in the missing data and decide for you if fully conditional specification or monotone method is more appropriate. You can also specify how many imputations you want to end up with (usually, a minimum of five imputations). Once multiple imputation has created several complete datasets, subsequent analysis will produce output for each imputed ‘complete’ dataset, plus a pooled output that estimates what the results might be, had there been no missing values. The statistical analyses, (e.g. t-test or logistic regression), are performed separately on each imputed data set, and the results are then combined into a ‘pooled’ or aggregated result. Due to the variation in the imputed values, there will also be variation in the parameter estimates, and appropriate estimates of standard errors and p values. There are, obviously, pitfalls to over-reliance on imputed data. The five imputations defaulted in many software is most likely not adequate to estimate stable p values. According to Graham et al. 6, a minimum of twenty imputations are required for 10 to 30% missing data, and over forty imputations if you have over 50% missing data. A general rule of thumb is that the number of imputations required should be similar to the proportion of cases with missing data; that is, if you have 19% of cases with missing data, you should perform at a minimum, twenty imputations 6. It has already been mentioned that imputation estimates missing values from the distribution of the available data. If your particular variable does not follow a normal distribution, a pragmatic approach may be to transform that variable to approximate normal distribution before imputation and then transform it back again afterwards. Problems also arise when the missing variables are categorical or binary, different software handle this differently and further research into this area is required 7. Finally, ‘missing at random’ and ‘missing completely at random’ is an assumption which justifies your decision to impute. Where complete case and multiple imputation analyses give completely different results, you should attempt to understand the causes of such differences, and this should be reported in publications. If you have any questions on methodology, please direct them to Ms Method Matters at msmethodmatters@gmail.com SC is statistical advisor to Anaesthesia. No other competing interests declared.