Assessing the robustness of meta-analyses to variability in the frequency and timing of postoperative delirium assessment – a novel methodological approach

Background Postoperative delirium (POD) is a fluctuating confusional state occurring early after surgery. Given its fluctuating nature, POD detection is influenced by the frequency and timing of screening. Sparse assessments potentially underestimate the true incidence reported in individual studies, subsequently influencing the weight of studies in meta-analyses, and the results and precision of these meta-analyses. Objective To develop a method to evaluate the robustness of meta-analyses to the variation in the frequency and timing of POD assessment across included studies. Methods We developed an expert-based scoring system to rate studies according to the confidence that a patient reported not to have POD in the study truly did not, based on the reported timing and frequency of POD assessments. We describe how to use the system to generate hypothetical event rates for studies if the frequency and timing of POD assessment were optimal and how to use these rates in a sensitivity analysis of meta-analyses including these studies. We illustrate the method applying it to two meta-analyses as work examples, one of observational studies and one of randomized controlled trials (RCTs). Results In our work examples, the expert-based score for the individual studies ranged from 33–77% for the observational and 22–70% for RCTs meta-analysis. Sensitivity analyses showed a change in the pooled effect size, heterogeneity, and precision compared to the original estimates. Conclusion We present a method to evaluate the robustness of a meta-analysis of studies having POD as an outcome, when there is variation across individual studies in the frequency and timing of POD assessment. The confidence score acts as a preliminary index of the robustness of meta-analyses. Through work examples, we showed that the robustness depends on the characteristics of individual studies included.