Key result
Experts rate ~73% of missed ACS telephone triage calls as unsafe, but agreement is poor.
Why the study?
Root cause analyses of serious adverse events in out-of-hours primary care often point to telephone triage errors, but are hampered by hindsight bias.
Does blinded expert assessment reliably identify unsafe telephone triage in primary care patients with chest discomfort?
Case-Control (n=135)
Blinded to outcome
Does blinded expert assessment reliably identify unsafe telephone triage in primary care patients with chest discomfort?
Absolute Event Rate: 73.3% vs 22.5%
p-value: p=<0.001
Blinded expert assessment of telephone triage for chest discomfort has poor inter-rater reliability, highlighting the difficulty and potential limitations of retrospective safety evaluations.
Poor inter-rater reliability cautions against relying on expert review for triage safety; leaves open its value in primary care chest discomfort evaluations.
OBJECTIVE: Root cause analyses of serious adverse events (SAE) in out-of-hours primary care (OHS-PC) often point to errors in telephone triage. Such analyses are, however, hampered by hindsight bias. We assessed whether experts, blinded to the outcome, recognize (un)safety of triage of patients with chest discomfort, and we quantified inter-rater reliability. STUDY DESIGN AND SETTING: This is a case-control study with triage recordings from 2013-2017 at OHS-PC. Cases were missed acute coronary syndromes (ACSs, considered as SAE). These cases were age- and gender-matched 1:8 with the controls, sampled from the remainder of people calling for chest discomfort. Fifteen experts listened to the recordings and rated the safety of triage. We calculated sensitivity and specificity of recognizing an ACS and the intraclass correlation. RESULTS: In total, 135 calls (15 SAE, 120 matched controls) were relistened. The experts identified ACSs with a sensitivity of 0.86 (95% CI: 0.71-0.95) and a specificity of 0.51 (95% CI: 0.43-0.58). Cases were rated significantly more often as unsafe than the controls (73.3% vs. 22.5%, P < 0.001). The inter-rater reliability for safety was poor: ICC 0.16 (95% CI: 0.00-0.32). CONCLUSIONS: Blinded experts rated calls of missed ACSs more often as unsafe than matched control calls, but with a low level of agreement among the experts.
No takes yet. Share an insight, caveat, or question.
Erkelens et al. (2020) conducted a case-control in Chest discomfort (n=135). Blinded expert assessment of triage recordings vs. Matched control calls was evaluated on Rating triage as unsafe (p=<0.001). Blinded expert assessment of telephone triage recordings rated missed acute coronary syndrome cases as unsafe more often than matched controls (73.3% vs 22.5%, P<0.001), but agreement was poor.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: