Using indicators to quantify the potential to improve the quality of health care

PURPOSE: Although clinical indicators allow individual providers to monitor and improve their own performance and quality of care, another important role for the indicators is to provide comparative information across all providers. We show that the 'league table' approach is ineffective, and provide an alternative method that uses the comparative rates to quantify the potential for improvement at both the provider and the national level. DATA SOURCES: The methods are applied to English and Australian hospital clinical indicators. METHODS: The key is to regard clinical indicators as screening tools that measure performance in one or more dimensions. All screening processes require explicit tests to determine whether the result should be classified as either positive (requires further investigation) or negative (requires continued monitoring). A clinical indicator will be defined as positive if any of the three following criteria are met: (1) large variation between all areas or hospitals, as defined by the 20th centile gains: requires improvement in the health care system; (2) large variation between strata (rural/urban, teaching/non-teaching, public/private, State): requires action in the relevant stratum; (3) outlier hospitals: requires quality improvement in the individual hospitals. Two techniques are used to determine whether any of the three criteria are positive: (1) empirical Bayesian estimation to calculate 'shrunken' rates; and (2) use of the 20th centile to quantify the potential gains or improvement. RESULTS: For 185 Australian indicators, 55 clinical indicators had system gains involving better outcomes for at least 1000 patients per indicator. Using a set of criteria and subjective judgement, we identified some key areas for quality improvement in Australia. CONCLUSION: Ranking of hospitals does not quantify the potential gains that could be achieved. Indicators that measure health care processes should be reported by quantifying the potential gains, thus encouraging action. Estimating the gains across many indicators allows priorities to be established, such as identifying the areas with the greatest potential for improvement. The main tasks are to then provide the tools and resources to tackle those areas with the most gains.