Key result
Analyzing individual surgeon mortality rates provides low statistical power, with median case volumes for colorectal resections yielding only 20% power to detect a mortality rate 3 times the national median.
Observational
Yes
Current case volumes for individual surgeons are statistically underpowered to reliably detect outliers with increased mortality rates across multiple surgical procedures.
Low surgeon-level power precludes reliable outlier detection; leaves open whether hospital-level or process metrics better inform quality monitoring.
OBJECTIVES: There is controversy on the proposed benefits of publishing mortality rates for individual surgeons. In some procedures, analysis at the level of an individual surgeon may lack statistical power. The aim was to determine the likelihood that variation in surgeon performance will be detected using published outcome data. DESIGN: A national analysis surgeon-level mortality rates to calculate the level of power for the reported mortality rate across multiple surgical procedures. SETTING: The UK from 2010 to 2014. PARTICIPANTS: Surgeons who performed colon cancer resection, oesophagectomy or gastrectomy, elective aortic aneurysm repair, hip replacement, bariatric surgery or thyroidectomy. OUTCOMES: The likelihood of detecting an individual with a 30-day, 90-day or in-patient mortality rate of up to 5 times the national mean or median (as available). This was represented using a novel heat-map approach. RESULTS: Overall mortality rates for the procedures ranged from 0.07% to 4.5% and mean/median surgeon volume was between 23 and 75 cases. The national median case volume for colorectal (n=55) and upper gastrointestinal (n=23) cancer resections provides around 20% power to detect a mortality rate of 3 times the national median, while, for hip replacement, this is a rate 5 times the national average. At the mortality rates reported for thyroid (0.08%) and bariatric (0.07%) procedures, it is unlikely a surgeon would perform a sufficient number of procedures in his/her entire career to stand a good chance of detecting a mortality rate 5 times the national average. CONCLUSIONS: At present, surgeons with increased mortality rates are unlikely to be detected. Performance within an expected mortality rate range cannot be considered reliable evidence of acceptable performance. Alternative approaches should focus on commonly occurring meaningful outcome measures, with infrequent events analysed predominately at the hospital level.
No takes yet. Share an insight, caveat, or question.
Harrison et al. (2016) conducted an observational in Surgical procedures. Individual surgeon case volume was evaluated on Likelihood of detecting an individual with a 30-day, 90-day or in-patient mortality rate of up to 5 times the national mean or median. Analyzing individual surgeon mortality rates provides low statistical power, with median case volumes for colorectal resections yielding only 20% power to detect a mortality rate 3 times the national median.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: