Consistency of retrospective triage decisions as a standardised instrument for audit.

OBJECTIVES: To determine the level of agreement between senior medical staff when asked to perform retrospective case note review of nursing triage decisions, both before and after development of a consensus approach. METHODS: Four medical reviewers independently allocated triage categories to 50 emergency department patients after review of their case notes. They were blind to the identity of the triage nurse and their triage categorisation. The process was repeated twice, firstly after agreement on a consensus approach and then using formal guidelines. RESULTS: Agreement between reviewers was initially fair to moderate (kappa = 0.27 to 0.53). This failed to improve after development of a consensus approach (kappa = 0.29 to 0.57). There was a trend towards better agreement when guidelines were used but agreement was still only moderate (kappa = 0.31 to 0.63). CONCLUSIONS: Audit of nurse triage categorisation by senior medical staff performing case note review has only fair to moderate consistency between reviewers. Use of this technique will result in frustration among those whose performance is being audited if they recognise inconsistency in the standard they are compared against.