Interobserver variability in colonoscopy quality assessment: a retrospective standardized multicenter video-based study

BACKGROUND AND AIMS: Colonoscopy quality assessment is essential for adequate bowel preparation and complete examination, yet even validated tools such as the Boston Bowel Preparation Scale (BBPS) remain partly subjective. We assessed interobserver variability in expert evaluation using a standardized multicenter video dataset. METHODS: This retrospective multicenter study included 64 anonymized complete colonoscopy videos from two academic centers. Eight experienced gastroenterologists independently evaluated recordings in randomly assigned pairs. Videos were assessed by five reviewer-pair combinations; individual reviewers evaluated between 10 and 33 examinations, and each pair assessed between 10 and 23 videos. Assessments included segmental and total BBPS scores, bowel preparation adequacy, and recognition of key anatomical landmarks: ileocecal valve, appendiceal orifice, hepatic and splenic flexures, and anal verge. Interobserver agreement was assessed using linear weighted Cohen's kappa for segmental BBPS scores, intraclass correlation coefficient (ICC) for total BBPS score, and Cohen's kappa with overall percent agreement for bowel preparation adequacy and anatomical landmark recognition. Because reviewer pairs varied across examinations, agreement measures were interpreted as pooled pairwise agreement across independent expert assessments. The Wilcoxon signed-rank test was retained as a complementary analysis for paired BBPS score differences. RESULTS: Significant inter-reviewer variability was observed in BBPS scoring. Differences were found for the right colon, transverse colon, left colon, and total BBPS score: 2.42 vs 1.91, p<0.01; 2.47 vs 2.11, p<0.01; 2.44 vs 2.22, p<0.05; and 7.33 vs 6.23, p<0.01, respectively. Overall, bowel preparation adequacy classification did not differ significantly, although discordant judgments occurred in 34% of examinations. Anatomical landmark recognition also varied, particularly for the appendiceal orifice and colonic flexures. CONCLUSIONS: Expert-based assessment may show clinically relevant variability despite standardized review conditions, supporting the need for more objective and reproducible approaches to colonoscopy quality control.