Introducing Levels of Evidence to the Journal

Orthopaedic surgeons have always based their clinical care on evidence. Surgeons use evidence to make decisions tailored to an individual patient's needs and circumstances. The primary sources of evidence for clinicians are studies published in the medical and surgical literature, such as The Journal of Bone and Joint Surgery. In June 2000, The Journal introduced the quarterly Evidence-Based Orthopaedics section 1. This section introduces orthopaedic surgeons to recent randomized trials relevant to the practice of orthopaedic surgery published in forty-two journals other than The Journal of Bone and Joint Surgery. Structured abstracts of these studies are published along with solicited commentaries to place the evidence into context. Beginning this month, The Journal is making an addition to its clinical articles. All such articles will include a Level-of-Evidence Rating . Levels of evidence are hierarchical rating systems for classifying study quality. Several systems for rating levels of evidence are available (minerva.minervation.com/cebm/docs/levels.html). The one chosen by The Journal has five levels for each of four different study types—i.e., therapeutic, prognostic, diagnostic, and economic or decision modeling. The Journal is accordingly modifying its Instructions to Authors: authors submitting articles must now clearly specify the primary research question of their study; they must classify the type of study as therapeutic, prognostic, diagnostic, or economic/decision analysis; and they must provide a Level-of-Evidence Rating of their approach to the primary research question. Every Level-of-Evidence Rating will be reviewed by the editors. The addition of the Level-of-Evidence Ratings to The Journal will have several benefits. Authors, reviewers, and readers will become familiar with the concept of levels of evidence, and studies will be improved by an explicit articulation of the primary research question. In addition, The Journal will be able to monitor and to periodically report trends in the quality of orthopaedic clinical research. Most important, the ratings will place a clinical research study into context for the reader. Higher levels of evidence should be more convincing to surgeons attempting to resolve clinical dilemmas 2. However, when using levels of evidence, readers need to consider several caveats. First, levels of evidence provide only a rough guide to study quality. In-depth assessment requires a critical appraisal of the specific study. Second, as randomized clinical trials are not always possible 3, Level-I evidence may not be available for all clinical situations. Level-III or IV evidence can still be of great value to the practicing orthopaedic surgeon. Finally, an answer to a clinical question must be based on a composite assessment of all evidence of all types. No single study provides a definitive answer. We look forward to your comments about this Journal initiative, feedback on the process, comments on its usefulness to orthopaedic surgeons, and debates about the Level-of-Evidence Ratings applied to individual studies. —James G. Wright, MD, MPH, FRCSC, Deputy Editor —Marc F. Swiontkowski, MD, Deputy Editor for Outcome Studies —James D. Heckman, MD, Editor-in-Chief - Levels of Evidence for Primary Research Question Types of Studies Therapeutic Studies—Investigating the Results of Treatment Prognostic Studies—Investigating the Outcome of Disease Diagnostic Studies—Investigating a Diagnostic Test Economic and Decision Analyses—Developing an Economic or Decision Model Level I 1. Randomized controlled trial, a. Significant difference, b. No significant difference but narrow confidence intervals, 2. Systematic review 2 of Level-I randomized controlled trials (studies were homogeneous) 1. Prospective study 1, 2. Systematic review 2 of Level-I studies 1. Testing of previously developed diagnostic criteria in series of consecutive patients (with universally applied reference "gold" standard), 2. Systematic review 2 of Level-I studies 1. Clinically sensible costs and alternatives; values obtained from many studies; multiway sensitivity analyses, 2. Systematic review 2 of Level-I studies Level II 1. Prospective cohort study 3, 2. Poor-quality randomized controlled trial (e.g., <80% follow-up), 3. Systematic review 2, a. Level-II studies, b. nonhomogeneous Level-I studies 1. Retrospective study 4, 2. Study of untreated controls from a previous randomized controlled trial, 3. Systematic review 2 of Level-II studies 1. Development of diagnostic criteria on basis of consecutive patients (with universally applied reference "gold" standard), 2. Systematic review 2 of Level-II studies 1. Clinically sensible costs and alternatives; values obtained from limited studies; multiway sensitivity analyses, 2. Systematic review 2 of Level-II studies Level III 1. Case-control study 5, 2. Retrospective cohort study 4, 3. Systematic review 2 of Level-III studies 1. Study of nonconsecutive patients (no consistently applied reference "gold" standard), 2. Systematic review 2 of Level-III studies 1. Limited alternatives and costs; poor estimates, 2. Systematic review 2 of Level-III studies Level IV Case series (no, or historical, control group) Case series 1. Case-control study, 2. Poor reference standard No sensitivity analyses Level V Expert opinion Expert opinion Expert opinion Expert opinion 1. All patients were enrolled at the same point in their disease course (inception cohort) with ≥80% follow-up of enrolled patients. , 2. A study of results from two or more previous studies. , 3. Patients were compared with a control group of patients treated at the same time and institution. , 4. The study was initiated after treatment was performed. , 5. Patients with a particular outcome ("cases" with, for example, a failed total arthroplasty) were compared with those who did not have the outcome ("controls" with, for example, a total hip arthroplasty that did not fail).