Systematic Review of the “Pragmatism” of Pragmatic Critical Care Trials

OBJECTIVES: To assess the pragmatism of published critical care randomized controlled trials self-described as pragmatic using a validated tool. DATA SOURCES: Medical Literature Analysis and Retrieval Online database and PubMed interface from inception to November 1, 2021. STUDY SELECTION: We performed a systematic search of randomized controlled trials evaluating interventions for critically ill adults that self-identified as pragmatic in title or abstract. DATA EXTRACTION: Reviewers independently performed study selection and data extraction in duplicate; discrepancies were resolved by consensus. Pragmatism was assessed independently in duplicate by trained reviewers using the Pragmatic-Explanatory Continuum Indicator Summary 2 (PRECIS-2), a validated tool designed to represent how explanatory/pragmatic a trial is on the pragmatic to explanatory continuum. Trials were scored in nine domains on a 5-point continuum (from 1 = very explanatory to 5 = very pragmatic). Discrepancies of greater than 2 points were adjudicated by consensus discussion. DATA SYNTHESIS: The search resulted in 284 studies; 56 met eligibility criteria. Forty-one of the trials had a discrepancy in at least one domain that required consensus discussion, most commonly in domains of eligibility and follow-up. Twelve studies (21.4%) were scored as “overall pragmatic,” defined as score of greater than 4 in five domains provided the scores in the remaining domains were three. The overall PRECIS-2 score of self-identified pragmatic studies increased from 1995 to 2021 suggesting increasing pragmatism over time. Pragmatic trials were more likely to have a waiver of informed consent ( p = 0.05). CONCLUSIONS: The number and pragmatism of self-identified pragmatic trials have increased, particularly in the past decade. However, less than one-quarter of these trials that use the term pragmatic in title or abstract were retrospectively rated as pragmatic. Our results support the concept that trials are designed on a spectrum of pragmatic to explanatory. Advances in the design and reporting of critical care trials are needed to ensure their real-world applicability.