pECGreview: Using “Not-to-Miss” ECG diagnoses to assess competency of trainees and faculty in ECG interpretation via a weekly online ECG education program

Figures

• 

  Figure 1. Screenshot of the pECGreview survey format. pECGreview = Pediatric ECG Review.

• 

  Figure 2. Distribution of mean scores as correct, OverDx, or UnderDx varied among the 3 scoring systems (P < .001). Group 1 accuracy scores were coded based on a subjective assessment of the accuracy of free-text responses as previously described.15 Those same free-text responses were also assigned 1 or more of the 18 NTM diagnoses or “None” (assignedNTM value). All 3 data sets were then assigned a correctness score of correct, OverDx, and UnderDx. The percent of correct scores is displayed in green, overdiagnoses displayed in yellow, and underdiagnoses displayed in red. Responses more often were scored as correct when using the accuracy scoring approach than the assignedNTM (P < .001) and NTM approaches (P < .001) (84.3%, n = 1164; 62.0%, n = 856; 50.0%, n = 910). Overdiagnosing occurred more frequently with use of the NTM approach than the assumed NTM scoring approach. Underdiagnosing occurred at similar rates in the NTM approach and assignedNTM approach (P > .4), but at a greater frequency than in the accuracy approach (18.8%, n = 348; 17.0%, n = 241; 6.3%, n = 94; P < .001). NTM = Not-to-Miss; OverDx = interpretations that erroneously included diagnoses that were not correct; UnderDx =…

• 

  Figure 3. A–C: Distribution of response scores by NTM ECG diagnosis, for 3 groups of scoring systems. 10 NTM diagnoses were represented across the 23 ECGs. Each diagnosis represents a single ECG except for none (12 ECGs), long QTc (2 ECGs), and VT (2 ECGs). The scores for each diagnosis are listed in order of increasing correctness for the group 1 accuracy scores. A: Group 1 accuracy scores had relatively high rates of correctness across all diagnoses, with “Atrial fibrillation” most frequently underdiagnosed. B: Group 1 assignedNTM response scores showed more frequent underdiagnosing, especially in NTM diagnoses of “Atrial fibrillation,” “AVB - high grade,” and “SVT.” C: Group 2 NTM response scores had similar distributions to those of group 1 assignedNTM. Across group 1 assignedNTM and group 2 NTM, ECGs with an NTM diagnosis of “None” were most frequently overdiagnosed. “Atrial flutter” and “ischemia” were most consistently diagnosed correctly in these groups. AV = atrioventricular; ECG = electrocardiogram; Fib = fibrillation; JET = junctional ectopic tachycardia; QTc = corrected QT; NTM = Not-to-Miss; OverDx = interpretations that erroneously included diagnoses that were not correct;…

• 

  Figure 4. Percentage correct by ECG difficulty group across scoring systems. ECGs were retrospectively classified into 3 “difficulty” groups determined by accuracy: (1) no NTM diagnosis, (2) difficult NTM diagnoses (“Atrial fibrillation,” “AVB - high grade,” “SVT”), and (3) other/easier NTM codes. The percentage correct was compared across scoring systems (group 1 accuracy, group 1 assigned NTM, and group 2 NTM). Group 1 assigned NTM and group 2 NTM scoring groups had significantly lower percentage correct than group 1 accuracy in the difficult ECG groups (19% and 20% vs 79%). AVB = atrioventricular block; Dx = diagnosis; ECG = electrocardiogram; NTM = Not-to-Miss.

• 

  Figure 5. Group 2 NTM scores by time sequential order of ECG. By observation, there was no obvious improvement in the fraction of correct NTM scores over time but this analysis is limited by the variations in ECG difficulty. ECG = electrocardiogram; NTM = Not-to-Miss; OverDx = interpretations that erroneously included diagnoses that were not correct; UnderDx = interpretations that did not identify the correct diagnosis.