From ‘reckless’ to ‘mindful’ in the use of outcome data to inform service-level performance management: perspectives from child mental health

In the recent report on patient safety in the National Health Service (NHS) in England, Don Berwick calls on the NHS to align the necessity for increased ‘accountability’ with the necessity to ‘abandon blame as a tool’ in order to develop a ‘transparent learning culture’.1 Sir Bruce Keogh, Medical Director NHS, and colleagues’ recent analysis of outlier hospitals based on mortality data marks a key step on this journey, but has led to high-profile debate about the risk of possible ‘reckless’ (Sir Bruce Keogh's term) use of data if appropriate parameters are not established.2 ,3 If these and other equivalent proxies for outcomes are to be used safely and effectively to support performance management and quality improvement in the ways envisioned by both Keogh and Berwick, it is crucial to establish clearly agreed operational procedures. Drawing on our experience of collecting and interpreting outcome data in the challenging context of child mental health across the UK, we suggest adoption of a MINDFUL framework involving consideration of m ultiple perspectives, i nterpreting differences in the light of current evidence base, focus on n egative differences when triangulated with other data, d irected discussions based on ‘what if this were a true difference’ (employing the 75–25% rule), use of f unnel plots as a starting point to consider outliers, appreciation of u ncertainty as a key contextual reality and the use of l earning collaborations to support appropriate implementation and action strategies. Any attempt to measure ‘impact’ of a service using a given ‘outcome’ is complex. The Keogh report acknowledges: “two different measures of mortality, HSMR [Hospital Standardised Mortality Ratio] and SHMI [Summary Hospital Level Mortality Indicator] generated two completely different lists of outlier trusts.” This was ‘solved’ by using both lists, but with a suggestion to move to …