Clarity of outcomes in medical education: do we know if it really makes a difference?

There has been longstanding concern, rightly, with the specification and measurement of outcomes of medical education in all of its stages. Educating members of the medical workforce to be competent and caring professionals is a serious business and the stakes are high. Defining appropriate and defensible outcomes and ensuring that they are attained is core business for all medical educators. There are 3 papers in this issue of the journal that focus on different aspects of the contemporary discussions about outcomes in medical education. Hays and Baravilala1 consider the application of global standards to an accreditation of the Fiji School of Medicine. This is an approach favoured by the World Federation of Medical Education (WFME).2 While the WFME methodology places an emphasis on medical schools engaging in the processes necessary to achieve the standards, the setting of international standards with an expectation that they will be achieved is a clear example of a focus on outcomes. Martin Talbot3 discusses a second form of outcome in his critique of competency models in graduate medical education. In this model, with its origins in industrial training, outcomes are defined in terms of specific technical competencies, which must be achieved before trainees move to the next stage. Charlotte Rees' paper is also a critique.4 She draws on insights from educational theory to examine the ‘outcomes-based’ approach to medical education advocated by Harden5 and others, which has gained increasing popularity in recent times. The essence of this work is the determination of broad and significant outcomes from which the educational programme is organised, using a process known as ‘designing down’ of the curriculum. But there has also been longstanding interest in process in medical education. This is perhaps best exemplified in the longevity and ubiquity of problem-based learning (PBL). While Rees does point out that the pre-specification of learning goals in PBL curricula lays this approach open to charges of paying ‘lip-service’ to process, the very rationale for PBL lies in the centrality of group functioning and learner self-direction. Advocacy of the importance of these learning processes has remained and has survived the claims of those who argue that existing research is unable to demonstrate the clear superiority of PBL in its knowledge outcomes for students.6–8 Debate about process and outcome is not new in the general educational literature. There was a particular interest in this debate in the UK curriculum literature in the mid-1970s. One of the influential participants and writers in this debate, Lawrence Stenhouse,9 likened concern with stating outcomes to setting the height of the bar for a high jumper. Athletes would improve their jumping performance not by having the bar set higher, but by examining, receiving feedback on and analysing their own jumping performance. The 3 papers in this issue highlight some of the perennial themes in the debate about process and outcomes. Hays and Baravilala's article raises the question of ‘whose outcomes’? Should outcomes be universal global standards or should medical schools look to more locally relevant and appropriate outcomes in a ‘symbiotic’ approach to curriculum development?10 Talbot's paper deals with the restricting nature of outcomes, particularly if written in the form of narrow competencies. The curriculum may be reduced to lower level competencies that are more easily defined and measured. Finally, Rees' paper brings up questions of power and control. The construction of curricula by expert designers according to pre-specified outcomes reduces the control of both teacher and learner over their own fundamental teaching and learning experiences. These arguments and other claims on both sides of the process/outcomes debate have not been tested in practice. Back in the 1970s Stenhouse advocated their testing in classroom-based research so that their real effects on teachers and students could be known.9 This has some similarities to the recent stand taken by Norman and Schmidt11 on research into PBL in medical education. They point to the serious limitations of outcomes-based research on PBL and argue for theory-based research investigating some of the fundamental questions in PBL, such as the effect of prior knowledge on learning, the quality of the problems used, tutor performance and group functioning. Thus if medical educators really want to contribute more than further rhetoric to the process/outcomes debate they should engage with the research questions and research methodologies that will examine how achieving clarity about outcomes really does affect what teachers and learners do. It is questions such as those below that require some research effort. How do outcome statements drive student learning? Is student learning driven more by statements of outcomes or statements of what is to be assessed? Do different types of outcomes (standards, competencies, broad outcome statements) drive student learning in different ways? Do locally, nationally or globally derived outcomes drive student learning in different ways? Do outcome statements encourage or discourage student direction in learning? How do outcome statements drive teacher activity, selection of content, selection of learning activities, assessment? Does the adoption of externally derived outcomes affect teacher and student engagement with the curriculum? Does participation in the process of determining outcome statements affect teacher and student engagement with the curriculum? This call for involvement with these and other similar research questions is not intended as a prescription for conservatism. It is not to suggest the suspension of activity and debate on the use of outcomes and consideration of their effects on the processes of teaching and learning at all levels in medical education. Rather, it is a call to ground the debate in the reality of those effects so that both process and outcomes can be harnessed to the effective education of competent and caring medical professionals wherever they are needed.