Key result
Preoperative mortality risk calculators show poor to moderate interrater reliability in high-risk noncardiac surgery.
Why the study?
It was unknown how online preoperative mortality risk calculators perform across different raters for elective high-risk noncardiac surgical patients.
Do the POSPOM, SRC, and SORT preoperative mortality risk calculators demonstrate reliable interrater agreement when used by anesthesiologists for elective high-risk noncardiac surgical patients?
Observational (n=34)
Do the POSPOM, SRC, and SORT preoperative mortality risk calculators demonstrate reliable interrater agreement when used by anesthesiologists for elective high-risk noncardiac surgical patients?
Effect estimate: ICC 0.46 (SORT), 0.65 (SRC), 0.63 (POSPOM) (95% CI 0.30-0.63 (SORT), 0.51-0.78 (SRC), 0.49-0.77 (POSPOM))
Preoperative mortality risk calculators (POSPOM, SRC, SORT) exhibit poor to moderate interrater reliability, suggesting they should be used with caution for clinical identification and counseling of high-risk surgical patients.
May warrant cautious use of these calculators for risk stratification; leaves open whether standardized training improves reliability.
BACKGROUND: Multiple preoperative calculators are available online to predict preoperative mortality risk for noncardiac surgical patients. However, it is currently unknown how these risk calculators perform across different raters. The current study investigated the interrater reliability of three preoperative mortality risk calculators in an elective high-risk noncardiac surgical patient population to evaluate if these calculators can be safely used for identification of high-risk noncardiac surgical patients for a preoperative multidisciplinary team discussion. METHODS: Five anesthesiologists assessed the preoperative mortality risk of 34 high-risk patients using the preoperative score to calculate postoperative mortality risks (POSPOM), the American College of Surgeons surgical risk calculator (SRC), and the surgical outcome risk tool (SORT). In total, 170 calculations per calculator were gathered. RESULTS: Interrater reliability was poor for SORT (ICC (C.I. 95%) = 0.46 (0.30-0.63)) and moderate for SRC (ICC = 0.65 (0.51-0.78)) and POSPOM (ICC = 0.63 (0.49-0.77). The absolute range of calculated mortality risk was 0.2-72% for POSPOM, 0-36% for SRC, and 0.4-17% for SORT. The coefficient of variation increased in higher risk classes for POSPOM and SORT. The extended Bland-Altman limits of agreement suggested that all raters contributed to the variation in calculated risks. CONCLUSION: The current results indicate that the preoperative risk calculators POSPOM, SRC, and SORT exhibit poor to moderate interrater reliability. These calculators are not sufficiently accurate for clinical identification and preoperative counseling of high-risk surgical patients. Clinicians should be trained in using mortality risk calculators. Also, clinicians should be cautious when using predicted mortality estimates from these calculators to identify high-risk noncardiac surgical patients for elective surgery.
No takes yet. Share an insight, caveat, or question.
Vernooij et al. (2024) conducted an observational in Elective high-risk noncardiac surgery (n=34). Preoperative mortality risk calculators (POSPOM, SRC, SORT) was evaluated on Interrater reliability (ICC 0.46 (SORT), 0.65 (SRC), 0.63 (POSPOM), 95% CI 0.30-0.63 (SORT), 0.51-0.78 (SRC), 0.49-0.77 (POSPOM)). Preoperative mortality risk calculators (POSPOM, SRC, and SORT) exhibited poor to moderate interrater reliability (ICC 0.46 to 0.65) for elective high-risk noncardiac surgical patients.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: