Inter‐ and Intra‐Rater Reliability of 4D ‐ CT Versus Bronchoscopy in Diagnosing and Grading Tracheomalacia

Objectives Bronchoscopy and 4D‐CT can be used to diagnose and characterize tracheomalacia. This study aims to assess inter‐ and intra‐rater reliability of rigid bronchoscopy and 4D‐CT imaging in determining tracheomalacia severity. Methods De‐identified rigid bronchoscopy videos and 4D‐CT derived airway segmentations were collected for 10 patients with varying degrees of dynamic tracheal collapse, for a total of 20 videos for review. A randomized, multidisciplinary survey was administered to physicians from otolaryngology, pulmonology, general surgery, and radiology at two institutions. Participants rated the severity of collapse for all cases at two time points, 1 month apart. Gwet's AC2 was used to assess inter‐ and intra‐rater reliability, with values ranging from 0 (slight agreement) to 1 (almost perfect agreement). Results Nineteen physicians completed the initial ratings (T 1 ), and 12 completed the repeat ratings 1 month later (T 2 ). Inter‐rater agreement for bronchoscopy was substantial at T 1 (AC2 = 0.79, 95% CI: [0.57, 1]) and at T 2 (AC2 = 0.76, 95% CI [0.52, 1]). Similarly, ratings of 4D‐CT segmentations demonstrated substantial agreement initially (T 1 : AC2 = 0.66, 95% CI [0.44, 0.89]) and at T 2 (AC2 = 0.68, 95% CI [0.39, 0.97]). Bronchoscopy (AC2 = 0.91, 95% CI: [0.87, 0.96]) and 4D‐CT (AC2 = 0.84, 95% CI: [0.76, 0.91]) had substantial agreement for intra‐rater reliability. Conclusion Endoscopic assessment showed higher inter‐ and intra‐rater reliability than 4D‐CT. Future studies should focus on defining the optimal role of these modalities to establish a standardized severity rating, thereby improving identification of surgical candidates. Level of Evidence 3.