Reliability and Validity of Scores on the Emergency Severity Index Version 3

Objectives: No widely used triage instrument accurately assesses patient acuity. The Emergency Severity Index (ESI) promises to facilitate reliable acuity assessment and possibly predict patient disposition. However, reliability and validity of ESI scores have not been established in emergency departments (EDs) outside the original research sites, and version 3 (v.3) of the ESI has not been evaluated. The study hypothesis was that scores on the ESI v.3 show good interrater reliability and predict hospital admission, admission site, and death. Methods: The authors conducted an ED‐based cross‐sectional retrospective study of 403 systematically selected ED records of patients who presented to an academic medical center. Twenty‐seven variables were abstracted, including triage level assigned, admission status, site, and death. Using a standard process, the researchers determined the true triage level. Weighted kappa and Pearson correlation were used to calculate interrater reliability between true triage level and triage score assigned by the registered nurse (RN). The relationships between the true ESI level and admission, admission site, and death were assessed. Results: Interrater reliability between RN ESI level and the true ESI level was kappa = 0.89; Pearson r = 0.83 (p < 0.001). Hospital admission by ESI level was as follows: 1 (80%), 2 (73%), 3 (51%), 4 (6%), and 5 (5%). A higher percentage of ESI level‐1 and level‐2 patients (40%, 12%) were admitted to the intensive care unit than ESI levels 3–5 (2%, 0%, 0%). Admission to telemetry for ESI levels 1‐5 was 20%, 19%, 7%, 1%, and 0%, respectively. Three of four patients who died were ESI level 1 or 2. Conclusions: Scores on the ESI assigned by nurses have excellent interrater reliability and predict hospital admission and location of admission.