Key result
Operators in the lowest-rated peer review tertile had higher rates of complications and major adverse cardiac events than the highest-rated tertile.
Why the study?
Reviewer disagreements about the standard of care raise concerns regarding the use of peer review for high-stakes assessments of physician competency.
Does case-based peer review of PCI operators demonstrate interobserver reliability and correlate with clinical outcomes?
Observational (n=3,390)
Blinded
Yes
Does case-based peer review of PCI operators demonstrate interobserver reliability and correlate with clinical outcomes?
Answers are grounded in this paper and cite their sources.
Case-based peer review of PCI operators shows moderate interobserver reliability and correlates with clinical outcomes, but reviewer disagreements on standard of care limit its utility for high-stakes competency assessments.
BACKGROUND: Case-based peer review of percutaneous coronary intervention (PCI) is used by many hospitals for quality improvement and to make decisions regarding physician competency. However, there are no studies testing the reliability or validity of peer review for PCI performance evaluation. METHODS: We recruited interventional cardiologists from 12 Veterans Affairs Health System facilities throughout the United States to provide PCI cases for review. Ten reviewers performed blinded reviews such that each case was reviewed twice. Cases were rated on a scale of 1 to 5 (with 5 being the best) for 6 care domains (Appropriateness, Lesion Suitability, Strategy, Technical Performance, Outcome, and Documentation) with a summary performance score calculated as the average of all domains. Separately, reviewers determined whether the standard of care was met. Interobserver reliability of the summary performance score was calculated using interclass correlation coefficient. We examined procedural complications and 30-day mortality and major adverse cardiac events for all PCIs performed by these operators from 2019 to 2022 when stratified in tertiles by summary performance score. RESULTS: Of the 65 cases provided by 13 operators, the mean summary performance score was 3.90 (SD=0.78) out of 5. The interclass correlation coefficient was 0.53, indicating moderate interobserver reliability. For 19 cases (29.2%), 1 reviewer indicated that the performance did not meet the standard of care; however, the second reviewer disagreed in all these cases. Average performance scores ranged from 3.35 to 4.38. Among the 3390 PCIs performed by reviewed cardiologists from 2019 to 2022, the lowest-rated tertile had higher rates of complications (2.9% versus 1.8%, P <0.01) and major adverse cardiac events (10.6% versus 8.0%, P <0.01) compared with the highest-rated tertile. CONCLUSIONS: Case-based peer review identifies variation in physician performance that is correlated with PCI outcomes. However, reviewer disagreements about the standard of care raise concerns about the use of peer review for high-stakes assessments of physician competency.
No takes yet. Share an insight, caveat, or question.
Doll et al. (2025) conducted an observational in Percutaneous coronary intervention (n=3,390). Lowest-rated tertile of peer review performance score vs. Highest-rated tertile was evaluated on Major adverse cardiac events (p=<0.01). Operators in the lowest-rated peer review tertile had higher rates of complications and major adverse cardiac events than the highest-rated tertile.
Synapse has enriched 10 related papers. Consider them for comparative context: