Inter-rater agreement of a newborn calf lung ultrasound scoring system

Background Lung ultrasonography (LUS) is a reliable and noninvasive tool for detecting pulmonary abnormalities in neonatal calves. Although scoring systems for clinical decision-making are well established in human neonatology, equivalent validated systems for newborn calves are lacking for the perinatal period (0-48 h). Hypothesis/Objectives Assess inter-observer agreement among veterinarians regarding a proposed lung ultrasound scoring system for newborn calves adapted from human neonatology score to detect pulmonary problems soon after birth. Animals Fifty thoracic ultrasound items (25 images and 25 videos) from calves born in commercial Brazilian dairy herds were evaluated by 14 veterinarians with skills in calf LUS from Brazil, Canada, France, and the United States. Methods After a 5-min instructional video, raters scored each item (0-4) based on lung aeration and consolidation patterns and graded media quality and diagnostic confidence on a 1-5 Likert scale. Agreement was assessed using raw percentage agreement (PA), Gwet’s AC2, Krippendorff’s alpha (α), and weighted Fleiss’ kappa (κ). Results Inter-rater reliability for the 0-4 LUS score was very good: PA = 0.957 (95% confidence interval [CI], 0.940-0.973), AC2 = 0.845 (95% CI, 0.781-0.909), α = 0.839 (95% CI, 0.827-0.852), and κ = 0.849 (95% CI, 0.736-0.903). Median quality and diagnostic confidence were 4 (interquartile range [IQR] = 4-5). When restricted to high-confidence loops (scores 4-5), agreement increased slightly (PA = 0.960; AC2 = 0.880; α = 0.860). Conclusions and clinical importance The high inter-observer agreement confirmed the reliability of the newborn calf LUS score. This score therefore potentially could be further validated to follow lung changes after delivery in neonatal calves.