Ultrasound Competency and Practice: What's in a Number?

Merriam Webster defines the term competency as "the ability to do something well enough to meet a standard."1 Upon first impression, this definition seems simple. However, what is meant by "well enough?" Additionally, who sets the "standard" and what is the process for setting the standard? Assessing competency in medical education demonstrates the challenge of applying this essential concept to medical education training and skill maintenance. Whether for initial or recertification processes, these decisions can be difficult and fraught with evidence-based, administrative, and practical issues. The recent decision by the American Board of Internal Medicine (ABIM) to regard its deeply thought-out and debated maintenance of certification process as flawed illustrates the challenge of competency assessment in the medical field.2 In this case, ABIM elected to suspend implementation of the plan, demonstrating the relevant challenges faced by an established discipline within the medical field. Three articles in this month's issue of Academic Emergency Medicine address competency assessment in one area of medical education: the teaching and use of point-of-care ultrasound (US) by emergency physicians (EPs).3-5 Since the first American College of Emergency Physicians (ACEP) guidelines were published in 2001, the field of emergency medicine (EM) has invested tremendous effort and resources into the teaching of point-of-care US to EPs.6 These landmark ACEP guidelines have been extremely useful in furthering point of care US use. The guidelines relied on clinical and educational research to support the wise structural concepts for this increasingly essential clinical skill. The guidelines have used the literature to support educational and administrative pathways that define specialty specific guidelines. The article by Hall et al.3 attempts to provide a mnemonic for the use of limited, focused echocardiography by EPs. The authors first state that EP-performed cardiac US is a unique skill set that is improperly defined even by existing attempts to characterize focused cardiac US.7 They use their extensive collective experience to present a streamlined approach to teaching focused emergency cardiac US with an easy-to-remember mnemonic, "The Five Es." This mnemonic helps to support a pathophysiologic basis to emergent cardiac US with characterizations of chamber size and function. This well-written article provides structural aids with a good review of the performance and interpretation of the perceived complexity of cardiac US. The article by Chiem et al.4 attempts to address the "well enough" aspect for US assessment. In this study, the authors demonstrate that novice practitioners can achieve similar results to experts when dealing with one US scan type. The use of B-lines on thoracic US has been shown to be useful in diagnosing patients with acute decompensated heart failure.8 Initial studies illustrating this technique were performed by practitioners judged to be expert in emergency US. Chiem et al. sought to expand this concept to a wider US user group, specifically their resident trainees. Their methods use a 30-minute lecture that teaches both image acquisition and technique. The target resident users were individuals who had already completed a 2-week dedicated US rotation. Although 30 minutes is impressively brief to learn a new skill, the majority of EM residents have completed a dedicated emergency US rotation, thus providing a foundation that would seem representative for graduating residents.9 The Chiem et al. article also demonstrates a limitation of assessing physician competency. Although the novice resident practitioners were trained "well enough" to meet the "standard" of the expert practitioners, neither group achieved admirable diagnostic accuracy for diagnosing acute decompensated heart failure. Additionally, the investigators did not demonstrate changes in patient clinical outcomes through the use of this technology. The article by Blehar et al.5 attempts to address the means by which we measure competency in US. The authors break down US education into image acquisition and image interpretation. A third component, using those two to guide clinical care, is not included in their review. They point out that many organizations use the number of US examinations performed as a surrogate for experience and, thus, for competency. Their findings demonstrate support for existing national guidelines, finding a plateau effect for skill acquisition. They note these plateaus for both image acquisition and interpretation for many studies to be in the double digits, if not exactly 25 to 50 US learning scans. Similarly, EM residents were able to learn point-of-care US "well enough" (95.9%) to meet expert "standards." Their results point to the concept that more experience should lead to better "competency," although a plateau effect will occur at some point. A limitation to this investigation is rendered in the form of a lack of demonstrated effects on patient outcomes, even if Merriam Webster's definition appears to have been satisfied. While inflection points may vary and appear higher than one might expect, the authors demonstrate that experience as defined by numerical goals may result in a much longer process than necessary to achieve competency. Guidelines produced by national bodies in support of defining competency in medical education use expert panels that rely on research published in peer-reviewed journals. When evidence is lacking, consensus opinion with reference to known practice and administrative processes are inserted. As a result, variable opinions are often proffered on questions such as the number of US scans needed for competency. In 2008, for example, the American College of Emergency Physicians recommended that physicians perform 25 to 50 US studies for each of the 11 core clinical applications.10 In contrast, the American Registry for Diagnostic Medical Sonography requires that physicians perform 800 US studies prior to taking a test to certify competency.11 The American College of Cardiology and the American Heart Association published recommendations in 2003 that learners perform 150 cardiac echocardiograms prior to independently performing echocardiography.12 The American College of Radiology requires that physicians interpret or supervise at least 500 US studies to certify competency.13 And finally, the American College of Chest Physicians suggest a portfolio of point-of-care US examinations ranging from three to 20, depending on the application.14 Confounding the issues, performance interpretation and clinical integration are not easily addressed by any of these guidelines. As the wide-ranging criteria for US competency illustrate, specialty groups will use a combination of evidence- and consensus-based processes without outcomes data to support a specific position for defining competency. Competency has and should move beyond simplified numerical goals. EM research in US is providing new answers to these challenging education and competency questions. Certainly, more research in emergency US needs to be done, with an emphasis on outcomes that include technique, education, competency assessment, and patient outcomes. Such an approach may lead to future guidelines with a more scientific, data-proven foundation for competency-defining recommendations.