Inter-rater reliability between inexperienced and experienced raters using a rabbit sedation scale

To assess the reliability of a published rabbit sedation scale when used by inexperienced raters compared with experienced raters. Experimental, randomized and blinded study. A group of 16 rabbits, 3–4 months old and 1.9 ± 0.5 kg (mean ± standard deviation). Rabbits were premedicated intramuscularly with midazolam 1 mg kg -1 and butorphanol 0.5 mg kg -1 ( n = 8) or midazolam 1 mg kg -1 and methadone 1 mg kg -1 ( n = 8). Following a short explanation of the sedation scale, two experienced raters (one Diplomate of the European College of Veterinary Analgesia and Anaesthesia, and one Diplomate of the European College of Zoological Medicine), and four inexperienced raters (veterinary students) applied the sedation scale before sedation, and 10, and 20 minutes after sedation. Weighted κ (κ w ) with linear weights and Gwet's agreement coefficient AC2 (Gwet’s AC2) were used to determine inter-observer agreement between the two diplomates, and Gwet's AC2 and Krippendorff's α (α k ) coefficients were used to evaluate inter-observer agreement between the four students for each of the eight items. Agreement was considered when κ w = 0.81–1.000. Lin’s concordance correlation coefficient was used to evaluate the concordance. A paired t-test and Gwet's AC2 coefficient were used to compare between groups. There was agreement within experienced [κ w = 0.981 (0.970–0.992), p = 0.001; Gwet’s AC2 = 0.973 (0.943–1.000), p = 0.001] and inexperienced raters [α k = 0.658 (0.618–0.699), p = 0.001; Gwet’s AC2 = 0.741 (0.705–0.778), p = 0.001]. There were statistically significant differences depending on the experience of the rater ( p = 0.001). The sedation protocol did not affect the agreement between raters ( p = 0.084). The rabbit sedation scale was reliable when used by experienced or inexperienced raters.