Big Versus Small Data and the Generalizability of the Rate of Computed Tomography Overuse in Minor Head Injury

In an electronic health record (EHR) chart review of adult emergency department (ED) trauma patients receiving a head CT from 2008 to 2013 within 14 community EDs, Sharp et al.1 estimate that approximately one-third of computed tomography (CT) scans in head injury are likely avoidable based on the Canadian CT Head Rule (CCHR). The analysis includes 27,240 adult trauma patients receiving head CTs and uses in-depth chart review of 100 random encounters with two independent abstractors (with high inter-rater reliability) to adjust the estimated number of avoidable CTs—the top Choosing Wisely initiative for emergency medicine.2 Overuse on the order of one-third is consistent with findings across the American healthcare system.3 Specifically, overuse on the order of one-third is well established for CT in ED patients with minor head injury. In 2012, we reported in a secondary analysis of data from a prospective, observational study, that 35% of CTs obtained in our urban, academic Level I trauma center ED were not recommended according to the CCHR.3 Since then, these findings have been confirmed in at least two other academic EDs.4, 5 The proliferation in publications during an era of increasingly limited research resources emphasizes the importance of establishing when descriptive studies have been adequately validated such that resources can be allocated to appropriate interventions. Sharp's findings demonstrate the generalizability of the known rate of overuse in this clinical scenario to the community setting. With that in mind, it begs the question of what value a “big data” approach adds to the knowledge in a well-described research area. This study lacks a suitable sample size justification. The authors state that they had “no prespecified power constraints … [and that] it was infeasible to perform chart review on tens of thousands of charts.”1 Without a power calculation, what makes the authors feel such a large sample size would be justified? Why not just report in more detail on the in-depth chart review of 100 random encounters with two independent abstractors with high inter-rater reliability? What specifically is the added value to this analysis of 27,240 patients when the random sample of 100 encounters had such high inter-rater reliability? After all, big data approaches have their own inherent limitations, as Wears and Williams6 so aptly put it: “The quantity and novelty of data cannot trump more fundamental issues of reliability, representativeness, bias, stability, construct validity, and context.” The authors’ adaptation of the CCHR to their analysis highlights many of these issues. The reader should note that the original CCHR study inclusion and risk criteria have been heavily adapted to the available EHR data with the justification that the authors have decided to use only “the three CCHR findings that are most influential in categorizing patient risk.”7 That said, there is value in EHR big data approaches such as the present one in monitoring CT overuse to maximize quality improvement initiatives. The emergency care research community should shift the use of limited research resources to address the clinical, nonclinical, and policy-relevant factors that contribute to overuse of CT in minor head injury.2, 8, 9