Inter-rater reliability of the South African Triage Scale: Assessing two different cadres of health care workers in a real time environment

Objective To estimate the inter-rater reliability of triage ratings within individual cadres of health care workers (HCWs) and between different cadres of HCWs using the South African Triage Scale (SATS). Methods Five final year medical students (FMSs), two enrolled nurses (ENs) and two Enrolled Nursing Assistants (ENAs), who had all been trained in the use of the SATS, were selected to prospectively triage Emergency Centre (EC) patients in real time. Twenty five patients were triaged twice on the first day by individual participants, and another 25 were triaged twice by a collaborative team on the second day. Quadratically weighted kappa (QWK) point estimates were calculated with 95% confidence intervals to assess agreement. Results For 25 patients analysed on day one, the QWK values were very high within professions triaging individually: among the FMSs (QWK = 0.94; 95% CI: 0.82–1.0), among the ENs (QWK = 0.92; 95% CI: 0.74–1.0) and moderate between the FMSs and ENs (QWK = 0.57; 95% CI: 0.33–0.81). For 25 patients analysed on day two a team of ENA and FMS, triaging collaboratively, demonstrated moderate agreement (QWK = 0.65; 95% CI: 0.46–0.85). Conclusion The inter-rater reliability of SATS ratings is excellent within individual HCWs, but significantly lower between different HCWs. This confirms previous reliability studies of the SATS using vignettes and if validated by larger studies would support the feasibility of further implementation of the SATS in primary health care settings across the Western Cape.