Making the case for personal professional monitoring in health care

The use of databases to identify both good and bad (acceptable and unacceptable) trends in the performance of simple and complex procedures in medicine has a long history and was advocated by such important medical and nursing practitioners as Florence Nightingale, Ernest Codman, Lord Moynihan and others [1–7]. Unfortunately, not all of these committed clinicians have gained the respect of their local medical colleagues [2,8,9]. Thus the need for objectively endorsed systems to monitor the occurrence of adverse clinical outcomes in day-to-day practice is paramount [10–12]. In pursuit of this goal the article by Spiegelhalter et al. in this issue is a welcome contribution [13]. The demonstration by the authors that the adverse events occurring in the practice of both the paediatric cardiac surgeons in Bristol and the general practitioner Harold Shipman could have been detected by routine data monitoring confirms earlier work arising from the Kennedy Inquiry and the Dame Janet Smith Inquiry [14–16]. Unfortunately, in the Bristol case, similar information had already been presented to the surgeons, anaesthetists and managers involved in the care of these patients with little impact on the service [17,18]. In fact a Cusum graph based on the same criteria employed by the Great Ormond Street paediatric cardiac surgery group had been presented to paediatric cardiac anaesthetists and surgeons well before the lethal series of operations was suspended [5]. The problem of professional performance monitoring remains one of objective review coupled with acceptance of performance data. This issue was clearly identified by Professor Ian Kennedy when he concluded in his inquiry: ‘Bristol was awash with data. There was enough information from the late 1980s onwards to cause questions about mortality rates to be raised both in Bristol and elsewhere had the mindset to do so existed’ [19]. Unfortunately the service was not suspended until 1995, by which time as many as 40 children may have died unnecessarily [14–16]. Kennedy goes on to identify the need for objective external criteria by which clinical services can be assessed. ‘The clinicians in Bristol had no one to satisfy but themselves that the service which they provided was of appropriate quality. There was no systematic mechanism for monitoring the clinical performance of healthcare professionals or of hospitals. For the future there must be effective systems within hospitals to ensure that clinical performance is monitored. There must also be a system of independent external surveillance to review patterns of performance over time and to identify good and failing performance’ [19]. The article in this issue demonstrates the applicability of such mechanisms to detect poor performance and the relative ease of their introduction [13]. The issue of professional or specialist group acceptance of this type of performance monitoring is addressed by adjustment of the ‘acceptable’ and ‘unacceptable’ adverse event rates and the allowable type 1 error (α) and allowable type 2 error (β) rates. Thus it would be reasonable to have a higher ‘acceptable’ adverse event rate for trainees than for specialists [20]. Exactly what these rates are will depend on clinical impressions, data from pilot studies, and early data collections [20]. The adverse event rates are likely to decline with monitoring as techniques, training and equipment improve [21]. However it should also be borne in mind that rates may increase as procedures become more difficult due to changing conditions such as age, obesity and illness severity [22]. Changing the ‘acceptable’ and ‘unacceptable’ adverse event rates has the effect of changing the distance between the horizontal lines on a Cusum plot. Similarly, changing the α and β values—that is, where α is the risk of incorrectly assigning the practitioner to an ‘unacceptable’ and β to an ‘acceptable’ performance level—will also change the vertical distance between the horizontal Cusum lines on the Cusum plot. Decreasing the α value will increase the distance between the lines of the Cusum plot. This will make it more difficult for a clinician’s Cusum plot to span a boundary interval and therefore more difficult for that clinician’s performance to be declared acceptable. In medical models the α and β limits have traditionally been set at 0.1 for ease of presentation of the data. If the α and β levels are the same, the horizontal Cusum lines are superimposed making interpretation of the graphs much simpler [5,20,23]. Why has such a mechanism not been introduced uniformly in clinical practice? Why are patients and their relatives still waiting for the widespread introduction of this type of monitoring process even though advocates of this process have made life-saving claims for its application [21]? One of the answers is clinician resistance, probably best enunciated in the early debate [22]. However, a degree of criticism must also be levelled at those health care administrators who have failed to ensure that such monitoring processes have been implemented. The necessary change in culture has sadly not occurred [7]. However, for the change in culture to be realised, it must be embraced by both clinicians and health care administrators for the benefit of all groups, including patients. One of the possible problems is the inability of clinical and administrative leaders in health care to believe that the type of monitoring proposed by Spiegelhalter et al. is possible and achievable [13]. This psychological block must be addressed if progress is to be made on this important issue in health care. The experience of the Geelong Personal Professional Monitoring Group—a group of anaesthetists at the Geelong Hospital in Australia committed to improving training and monitoring methods for all medical specialties—has been that cultural change in anaesthetic trainee attitudes to data collection and performance monitoring can be achieved provided data collection is made easy, feedback is rapid and informative, the training environment is supportive and the system is robust [6,7,23,24]. The optimistic message that the pilot study by Spiegelhalter et al. must convey to all the leaders of health care groups is that not only can personal professional monitoring be successfully and easily introduced into postgraduate training programmes but that it will also be enthusiastically and continuously employed by trainees in these programmes [6,7,23]. The exciting prospect is that these trainees will become the specialists of the future and will very rapidly help to create a changed culture and different role models for the medical students and trainees of the future [25]. When this has been achieved the spectres of Bristol and Harold Shipman will truly have been laid to rest. The challenge is to achieve widespread implementation of these processes in undergraduate and postgraduate training now.