Single measures of performance do not reflect overall institutional quality in colorectal cancer surgery

OBJECTIVE: To evaluate overall performance of English colorectal cancer surgical units identified as outliers for a single quality measure--30 day inhospital mortality. DESIGN: 144,542 patients that underwent primary major colorectal cancer resection between 2000/2001 and 2007/2008 in 149 English National Health Service units were included from hospital episodes statistics. Casemix adjusted funnel plots were constructed for 30 day inhospital mortality, length of stay, unplanned readmission within 28 days, reoperation, failure to rescue-surgical (FTR-S) and abdominoperineal excision (APE) rates. Institutional performance was evaluated across all other domains for institutions deemed outliers for 30 day mortality. Outliers were those that lay on or breached 3 SD control limits. 'Acceptable' performance was defined if units appeared under the upper 2 SD limit. RESULTS: 5 high mortality outlier (HMO) units and 15 low mortality outlier (LMO) units were identified. Of the five HMO units, two were substandard performance outliers (ie, above 3 SD) on another metric (both on high reoperation rates). A further two HMO institutions exceeded the second but not the third SD limits for substandard performance on other outcome metrics. One of the 15 LMO units exceeded 3 SD for substandard performance (APE rate). One LMO institution exceeded the second but not the third SD control limits for high reoperation rates. Institutional mortality correlated with FTR-S and reoperations (R=0.445, p<0.001 and R=0.191, p<0.020 respectively). CONCLUSIONS: Performance appraisal in colorectal surgery is complex and dependent on stakeholder perspective. Benchmarking units solely on a single performance measure is over simplistic and potentially hazardous. A global appraisal of institutional outcome is required to contextualise performance.