Revisiting ‘Assessment of clinical competence using an objective structured clinical examination ( OSCE )’

Editor's note: As part of our 50th volume celebrations, Medical Education is looking back at its most impactful articles, as defined by citation count. The most cited articles from each 5-year interval were identified and the original authors of one of them (or other knowledgeable scholars if the original authors could not be found) were asked to comment on the state of the field at the time of publication, the impact of the article, and what we have learned since then. The article illustrated in Figure 1 was one of the most cited articles in our journal in the 1977–1981 period. To see the other top-cited articles from Volumes 1–50, please view the interactive PDF by visiting www.mededuc.com. The Association for the Study of Medical Education (ASME) publication ‘Assessment of clinical competence using an objective structured clinical examination (OSCE)’1 provided the first complete description of the use of the OSCE to assess a student's clinical competence (Fig. 1). This current paper describes the background to the introduction of the OSCE and how it became the reference standard for performance assessment. In the late 1960s and early 1970s, as a senior lecturer in medicine in Glasgow and later in Dundee, I was responsible for student assessment. Three things struck me. The first was that the assessment of a student's clinical skills was regarded as important and a student could not graduate without passing the clinical examination. Secondly, there were major deficiencies in the clinical examination represented by the impact of the luck of the draw on both the type of patient seen by the student in the long case and the two examiners assigned to assess the student's competence. In the traditional clinical examination, the marks awarded by one examiner often varied considerably from those awarded by another examiner observing the same performance.2 John Stokes, an experienced examiner, described the clinical examination as the ‘sacred cow of British medicine’ and as a ‘half-hour disaster session’.3 Thirdly, it seemed to me that it should be possible to construct an examination that reliably assessed the range of competencies expected of the student, in which what was to be assessed at each station would be defined clearly in advance and reflected in a checklist and rating scale to be completed by the examiner. I was aware of the work of Barrows and Abrahamson4 and others on the use of simulated patients and felt that in some areas, such as the assessment of communication skills, a simulated patient could replace a real patient in the examination. In other situations, such as those concerning a patient with a hernia or goitre, the student should be assessed with a real patient. In the traditional clinical examination, marks awarded by one examiner often varied considerably from those awarded by another observing the same performance In Dundee I found a culture that encouraged innovation in medical education. I recall a conversation in the hospital car park with Alfred Cuschieri, who had been appointed professor of surgery. He made clear his determination to change the format of the final examination in surgery. Working with his two senior lecturers, Paul Priece and Robert Wood, and with Fergus Gleeson, who had come across from Ireland to work with me in the field of medical education, we planned to introduce the OSCE as the final examination in surgery at Dundee. The faculty board agreed that we could run a pilot final surgery OSCE alongside the traditional final surgical clinical examination. The result was a success and the following year the board agreed to replace the traditional surgery final clinical examination with an OSCE.5 This was possible because, in the UK, final examinations are the responsibility not of a national body, but of each school independently. It should be possible to construct an examination that reliably assessed the range of competencies expected of the student This early development of the OSCE has been described in more detail.6-9 A preliminary report that described the OSCE concept was published in the British Medical Journal10 and a more complete description was published in Medical Education as an ASME medical education booklet.1 The OSCE became widely adopted as an examination tool with which to assess students’ clinical competence. Teachers became aware of the approach through the published papers, and external examiners from other schools who participated in the Dundee OSCE spread the initiative to their schools, as did Dundee staff when they transferred to other schools. Ian Hart, a senior physician in Ottawa, Ontario, Canada, with whom I had previously collaborated in the area of thyroid research, spent a sabbatical in Dundee and rapidly became a convert to the OSCE. Together we organised the first Ottawa Conference in 1985, which aimed to share across the Atlantic approaches to the assessment of clinical competence, including the OSCE. The OSCE is now used in countries around the world to assess clinical competence in a range of disciplines, in different health care professions and in the different phases of education. It has also been used outside medicine, for example in the police force.8 More than 1600 papers on the OSCE have been published, including about 400 since 2011 (almost one new paper every 3 days!). Why has the OSCE been widely adopted as the recommended approach for the assessment of clinical competence and become the reference standard for performance assessment?11 Schneider12 identified four characteristics that lead to the adoption of an innovation. The first characteristic is perceived significance. Ideas that are adopted, Schneider argues,12 stand out not necessarily because they are but, rather, because they seem to be significant. The OSCE was perceived by teachers as addressing an important problem: the assessment of a learner's clinical competence. The second characteristic is philosophical compatibility: teachers must view the innovation as appropriate for use. Clinical teachers and examiners easily identified the OSCE with their own thinking. Schneider's12 third characteristic refers to occupational realism: ideas must be practical and easy to put into immediate use. This is certainly true of the OSCE. The fourth characteristic is transportability: the approach must be easily explainable to a busy colleague and adaptable for use in different situations. The OSCE has proved to be user-friendly and can be adapted for use in different contexts.8 Its characteristics of perceived significance, philosophical compatibility, occupational realism and transportability have facilitated the wide adoption of the OSCE as a tool to assess clinical competence. The OSCE is now used in countries around the world to assess clinical competence in a range of disciplines Reflecting on my experience with the OSCE over the last 35 years, I find I have learned eight lessons. Firstly, as demonstrated with our initial implementation of the OSCE, obtaining agreement for a pilot test is a quick way of introducing a new assessment approach. Secondly, having powerful champions is vital. The support of senior professors within the school of medicine was important and facilitated the introduction of the OSCE. Thirdly, there are major advantages if a medical school has the freedom to innovate in assessment. The medical school in Dundee had the authority to design its own assessment procedures and was not dependent on the agreement of a national examination body. Fourthly, flexibility and the ability to adapt a method to local contexts are key to the success of an innovation. Fifthly, scarce resources and the presence of large numbers of students need not stand in the way of innovation. I have yet to see an example of a situation in which the cost or the number of students to be assessed prevents the adoption of the approach. The only limitation is the imagination of the developer. My sixth lesson refers to the discovery that here are ‘good’ OSCEs and ‘not so good’ OSCEs. Reliability and validity are related to how the OSCE is implemented. The OSCE is really a POSCE (potentially objective structured clinical examination). There are ‘good’ OSCEs and ‘not so good’ OSCEs. Reliability and validity are related to how the OSCE is implemented. My seventh lesson reflects the knowledge that the clinical teacher is in a good position to advance medical education. I was a senior lecturer in medicine when I started the work on the OSCE. Although there is a place in medical education for large-scale research projects, more attention needs to be paid to the teacher as an action researcher. Although there is a place in medical education for large-scale research projects, more attention needs to be paid to the teacher as an action researcher. Finally, an approach to education will continue to evolve with time. Although the basic principles of the OSCE are as true today as when they were first described in the paper published in Medical Education in 1979,1 we can see developments in the use of technology to support the OSCE, such as in the use of simulators and in new automated marking schemes using, for example, iPads. Further, more serious attention is now paid to standard setting and to the assessment of competencies, such as teamwork skills and error management, and to patient safety, none of which featured much on the agenda in 1979.