Prospective evaluation of residents and nurses as severity score data collectors

OBJECTIVES: To determine the interobserver reliability of residents and nurses collecting Acute Physiology and Chronic Health Evaluation (APACHE II) data and the subsequent effect of these data collections on individual patient mortality prediction. DESIGN: In a prospective study, residents and nurses independently collected data to derive APACHE II scores. When their scores differed, a standard score was determined by one of the investigators. SETTING: A general medical and surgical ICU. PATIENTS: A total of 120 consecutive patients were included; of these patients, 79 had standard scores determined because resident and nurse scores differed. MAIN RESULTS: There was overall agreement between the residents and nurses with no significant difference between mean APACHE II scores or mean predicted mortality rates. Intraclass correlation coefficients confirmed good overall agreement between observer groups for predicted mortality rate: resident vs. nurse r2 = .94, resident vs. standard r2 = .94, and nurse vs. standard r2 = .90. However, clinically significant lack of agreement was demonstrated in 5% of the patients by the 95% confidence limits of agreement: resident vs. nurse -14 to +14%, resident vs. standard -10 to +14%, and nurse vs. standard -14 to +20%. CONCLUSIONS: While interobserver variability between resident and nurse data collection has minimal effect on derived predicted mortality rate with large patient groups, significant variability may occur in individual patients. Residents were more accurate data collectors than nurses.