Answers are grounded in this paper and cite their sources.
Evaluating whether a resident is qualified to graduate from an accredited orthopaedic surgery residency program remains a major educational challenge. To this point, residency programs have lacked robust approaches to measure resident performance. In response, the Accreditation Council on Graduate Medical Education (ACGME) is introducing the Next Accreditation System (NAS) for better documentation of residency programs and resident performance [5, 8]. As part of this introduction, all specialties will incorporate the six general competencies (medical knowledge, systems-based practice, patient care, communication and interpersonal skills, professionalism, and practice-based learning and improvement) into “milestones” to gauge resident performance at various levels of training. This system is designed to evaluate residents, assess programs, and assure the public that graduates of accredited residencies are safe, competent, practicing orthopaedic surgeons. The goal of this article is to outline the NAS, describe how it will affect teaching programs, and define what faculty members need to know before using the system. Background and Description of Milestones In 2011, a working group of eight orthopaedic surgeons—one from each subspecialty—developed reporting tools for orthopaedic resident performance [8]. The classifications ranged from beginning resident (Level 1) to the more advanced graduating resident (Level 5) (Fig. 1). The group of orthopaedic surgeons developed a list of sixteen milestone topics representing a cross-section of orthopaedic surgery, patient care evaluation, and orthopaedic knowledge (Table 1).Fig. 1: The ACGME requested competency-based reporting tools based on five levels of proficiency, ranging from novice resident (Level 1) to advanced graduating resident (Level 5). Reprinted with permission from the Accreditation Council for Graduate Medical Education and American Board of Orthopaedic SurgeryTable 1: Milestone topics for patient care and medical (orthopaedic) knowledgeWhat Will Change? The ACGME mandated that residencies form a clinical competency committee to determine the competency of each resident using the orthopaedic surgery milestones [8]. The clinical competency committee should be composed of specialties whose expertise includes the milestones topics. The committee will use the milestones criteria to provide consistent resident performance data to the ACGME. Every 6 months, residency programs will upload deidentified resident data on resident performance to the ACGME website. A program does not have to change their assessment tools or methods currently in place, but the clinical competency committee must translate assessments into the milestones criteria for every resident. For example, most programs use a standardized end-of-rotation evaluation of residents by faculty members. These forms are usually competency-based. The clinical competency committee will apply such information to the milestones assessment for a resident. Schedule Orthopaedic surgery is one of seven specialties—diagnostic radiology, emergency medicine, internal medicine, neurological surgery, pediatrics, and urology being the other six—in phase 1 of implementing the NAS [1]. Phase 1 specialties began using milestones criteria in July of this year with an initial upload of the first 6 months of data in December. Other programs, such as general surgery, will begin using the milestones in July 2014. Residency programs will collect data that will be reported to the ACGME as milestones every 6 months for all residents in the program. The first reporting will begin in December. Data collected from July through June 2014 will be reviewed with reassessment of the milestones in the fall of 2014. Limitations of the Next Accreditation System Current resident assessment tools may be inadequate to assess resident performance using the milestones criteria [2-4]. The ACGME listed operative procedures a graduate should be capable of performing (Table 2), such as a primary total hip and total knee replacement [8]. It remains unclear whether large-scale data collection supports the use of either the selected topics or the behavior anchors established for the milestones [8].Table 2: Operative procedures listed in the ACGME milestonesAlthough the goal of determining operative competency seems appropriate, the use of operative assessment tools to document resident performance requires review. The use of a global assessment tool at the end of rotation will not provide the same clarity as a specific instrument used at the time a procedure is performed. Ideally, a separate tool for operative evaluation of resident performance should also be developed for procedures listed in the milestones [6, 7]. The use of milestones requires additional paperwork and a new committee within each department that supports a residency. This requirement is an additional administrative burden with added cost. No additional financial support to residencies to facilitate implementation of the Next Accreditation System is expected to be forthcoming. What Do I Need to Do? Orthopaedic educators should be familiar with the milestones criteria in order to provide the best assessment for their residents and residency program. Residency programs are required to establish a clinical competency committee with oversight power of both resident assessment and evaluation using the milestones criteria. This group must achieve consensus regarding resident data for upload to the ACGME. Those who teach residents should be familiar with the topics that are germane to their subspecialty area. For academic programs that include subspecialty rotations, representatives from each rotation should be present, as well as provide evaluation data of residents from each of their respective rotations. For example, residents rotating on the hand service would be evaluated in the generic areas of systems based practice, communication and interpersonal skills, practice based learning and improvement, and professionalism. They would also be evaluated in topics generally found in hand surgery (carpal tunnel syndrome and distal radius fractures) for patient care and medical knowledge. Residency programs should review the milestone criteria, including how the criteria are used and how the data is collected. These tasks may be accomplished through routine faculty development sessions. Although assessment methods currently in place for a residency program may be used, the development of other assessment tools (such as operative assessment) is likely. In its nascent form, the milestones criteria seem to provide better and more consistent resident assessment, which may help programs with promotion decisions, particularly for the marginal resident. However, like all new programs, a period of trial and review will likely be necessary in order for the milestones to become a reliable system.
No takes yet. Share an insight, caveat, or question.
Paul J. Dougherty (2013) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: