When small degrees of bias in randomized trials can mislead clinical decisions: An example of individualizing preventive treatment of upper gastrointestinal bleeding

OBJECTIVE: Although randomized trials yield less biased estimates of treatment effects than other study designs, unconcealed randomization and lack of blinding can lead to overestimates of the treatment benefit on the order of 15% to 40% on average. In applying the results of clinical trials to patient care, clinicians need to be concerned about bias of this order when it would change patient management. The aim of this study is to assess under which circumstances clinicians need to be concerned about bias in clinical trials. DESIGN: Sensitivity analysis. SETTING: Recently published meta-analysis of RCTs on the benefit of H2-blockers on the prevention of gastrointestinal bleeding in critically ill patients. INTERVENTIONS: Assessment of the effect of different degrees of bias on clinical decision making in various clinical scenarios. RESULTS: Bias of even a modest degree (15% to 40% overestimation of effect) changed clinical decisions more frequently when treatment benefits were small and when the patients' risk to suffer an adverse outcome was low. When the benefit was large and the patients were at high risk, clinical decisions remained unchanged, despite bias in the estimation of effect. CONCLUSIONS: When treatments of moderate benefit are applied to patients of low to moderate risk, even small biases in the estimation of effects carry a high risk of erroneous decisions. When the treatment benefit is large and the patients are at high risk, clinicians need to be less concerned about bias in RCTs.