Standards for simulation

In many airlines a pilot qualified to fly a Boeing 737 can be trained and licensed to fly paying passengers in an Airbus A300 and be fully accredited for landing this aeroplane at airports around the world without ever leaving the ground [1-3]. One caveat is a requirement for twice yearly training and assessment sessions; failure implies immediate suspension from flying. This suspension is less draconian than it may seem because intensive retraining is provided promptly, with the opportunity for assessment. The reassessment is almost always passed, and it is hard to argue with the principle of ensuring that pilots are competent in the key skills of their trade. All of this is achieved through simulation. Consider, by contrast, the situation in anaesthesia. The oversight of continued professional development tends to be light-handed, with little emphasis on ensuring that core competencies are in place, and it seldom involves high-stakes assessment of the kind described above. It is only when something goes wrong that the competence of a doctor is likely to be brought into question – and at that stage not only is the response too late but the processes are typically punitive and destructive. Society’s continuing appetite for blame is illustrated by the extraordinary increase in manslaughter charges arising from simple errors in medical practice in the UK over the last decade [4, 5]. Little of this is surprising when one considers the traditional apprenticeship method of training in medicine (anaesthesia being no exception). Junior doctors were expected to ‘see one and then do one’ hoping not to ‘harm one’ before, all too soon, becoming the person to ‘teach one’. The clinical ‘curriculum’ was determined by the random presentation of patients to the unit on any particular day. More recently it has been recognised that something better is needed than simply throwing trainees into the whirlpool of acute medicine and hoping for the best. This has certainly led to more structured curricula and more explicit separation of training from service. (Imagine the response if the announcement on take-off was that the pilot was a ‘trainee’, and that supervision would be ‘distant’ or ‘remote’). The emphasis on supervision and on the provision of consultant-led patient care has increased dramatically. Nevertheless, the unpredictability of medical practice may challenge even an experienced consultant. It is difficult to gain and then to maintain competence in all aspects of practice. The problem lies not just in retaining in infrequently needed knowledge and skills; it lies also in the rapid pace of medical progress. Imagine now what the application of simulation could potentially mean in anaesthesia, applied at the level expected in aviation. No one method of training fits all needs, but imagine the gains in the quality and safety of our patient care that could be achieved through enhancing the established theoretical and clinical strands of anaesthesia training by introducing a planned curriculum of properly constructed simulation-based educational modules. Because simulation allows one to set the ‘clinical’ agenda, the educational objectives of each session could be pre-defined and the measures used in assessment could be chosen to drive learning towards these objectives. It would be possible to ensure that anaesthetists were competent to handle most relevant conditions, from common to rare but life-threatening. To some extent this is already happening. In healthcare, anaesthetists have been early to adopt simulation for training. From early beginnings with resuscitation manikins of the 1960s, anaesthetists have gone on to develop a range of more sophisticated simulators [6, 7]. The benefits of simulation in the education of anaesthetists is now fairly widely accepted [8-10], with enthusiasts even referring to simulation-based medical education as an ‘ethical imperative’ [11]. The United States Food and Drug Administration, the American College of Surgeons, the American Council for Graduate Medical Education, and the American Board of Anesthesiologists now all require some aspect of simulation-based training for a number of their qualifications [12]. Many simulators are now available and several societies have been established to nurture the expanding body of knowledge and professional activity related to simulation in anaesthesia [13]. Nevertheless, the use of simulation in healthcare lags far behind other high-risk industries, particularly aviation and the nuclear industry [14-18]. Uptake of simulation-based training is still patchy, especially in the context of the continuous professional development of specialist anaesthetists. However, the validity of simulation for training and, in particular, for assessment remains controversial [12, 19-23]. Few anaesthetists in a survey of 60 centres agreed with the use of simulation for maintenance of certification [24] and the majority of anaesthetists surveyed in Australia and the USA were opposed to making simulation-based testing compulsory [25]. Even the well-established Effective Management of Anaesthetic Crises (EMAC) [26] courses run under licence from the Australian and New Zealand College of Anaesthetists (ANZCA) include no summative assessment. If instructors observe participants whose performance in this course is worrying, they have little option but to allow them to return to practice with no remedial action. There are several good practical and theoretical justifications for excluding assessment from this course but this approach is nevertheless in stark contrast to the situation in aviation, as outlined above. Why this difference, given many obvious parallels [26, 27]? Several reasons come to mind. The number of simulation manufacturers in anaesthesia is limited, and the market is relatively small (notably in contrast to the market for advanced life support training). Anaesthesia shares only some similarities with aviation; the differences are substantial, not the least being the extraordinary variability between patients – even apparently similar patients, and even the same patient at different times. The weather may be unpredictable, but aeroplanes follow the principles of physics, while patients (it often seems) tend to be laws unto themselves [9, 28, 29]. The difficulty of capturing this variability in simulation is one of several reasons for the pervading uncertainty about the adequacy of available simulators in representing patients. Furthermore, evidence to demonstrate the effectiveness (particularly cost-effectiveness) of simulation in the training of health professionals is relatively sparse [11, 30, 31], and even more so in relation to their assessment. A more worrying basis for scepticism is expressed in the following question: does performance in ‘anaesthetising’ the simulators we have today reliably reflect performance in the clinical situation? Perhaps, perhaps not; or perhaps the answer depends on the context and the skills in question. This uncertainty is one of several reasons for the hesitancy mentioned above in relation to assessment in simulation courses. However an even more fundamental uncertainty in respect of simulation arises from the almost complete absence of agreed standards for any aspect of simulation-based education in healthcare. This, we think, is a major and very much under-appreciated barrier to realising the full potential of simulation in anaesthesia. Perhaps the most striking single difference between simulation in anaesthesia and simulation in aviation is the fact that in the latter it is underpinned by a clear understanding that all simulators have met well-defined and widely accepted standards (and the same can be said of simulation in the nuclear industry) [32, 33]. These standards, which are regularly updated [34], promote reproducibility and define benchmarks. They make it possible for a purchaser or user of a simulator to know what to expect and what not to expect and give the user confidence in the results they obtain when using that simulator. In 2006, the Society for Simulation in Healthcare held a Simulation Summit in Chicago, IL. Fifty-five individuals participated, representing 33 organisations. They identified the need for standards and guidelines as one of the most important requirements for reaching the full potential of medical simulation [35], yet there has still not been a concerted effort to establish such standards. Some organisations have made a start in this process. For example, ANZCA have defined guidelines for the simulator, the instructors and the environment to be used in their EMAC course (see their website: http://www.anzca.edu.au). Those who wish to provide the EMAC course must have their centres accredited in accordance with these guidelines and re-examined at least every 5 years. This is an important step toward the establishment of standards, but the guidelines give few, if any, quantitative criteria by which to evaluate a particular centre or simulator. But is there a problem? Surely we can simply depend on manufacturers to ensure that the simulators perform to specification? We thought so. Our department recently purchased a new high-end simulator. We were surprised to discover that it appeared to suffer from coarctation of the aorta (that is to say, the pulse at the right wrist was full and bounding while that at the left wrist was weak and thready). We were even more surprised (and initially seized by clinical self-doubt) when we found that the simulator routinely died when we gave it what we believed to be appropriate doses of isoflurane. To be clear, these were specific problems in a very sophisticated, complex and valuable educational tool, but they were not the only ones. Could we say, then, that our simulator failed to meet specifications? There indeed lay the rub – there weren’t any performance specifications. The company has been very helpful in addressing these problems, and we commend them for that, but surely there should be documentation defining the parameters for acceptable responses to particular inhaled concentrations of isoflurane, the volume and pressure of the pulses at the wrist, and many other aspects of how a simulator should look and function? We believe that standards for simulators, and for simulation centres, courses and instructors, described in quantitative terms and requiring documentation of compliance, are overdue. The specification, implementation, maintenance, and enforcing of standards will not be a trivial task. It will require the cooperation of members of academic institutions and leaders of industry. Organisations such as the Society in Europe for Simulation Applied to Medicine (SESAM) [36] and the Society for Simulation in Healthcare (SSH), as well as ANZCA, the Royal College of Anaesthetists and the Association of Anaesthetists of Great Britain and Ireland, have roles to play in this regard. It will take time, but it is within the grasp of these groups, working collaboratively, to establish a set of standards for anaesthesia simulators. This will be a very important step towards realising the full potential of simulation in anaesthesia – and this potential is very far-reaching. Anaesthetists and their patients need to be confident that they can correctly manage the full range of critical events they may face in practice. A major step towards assuring this confidence would be regular training in which core skills are both effectively taught and reliably tested in simulation centres in which all concerned can be confident that the simulators, the trainers and the assessments conform to the standards required for this purpose. The authors acknowledge constructive comments from Dr Jane Torrie, Dr Lara Hopley, and Kaylene Henderson.