Observer agreement in the grading of oral epithelial dysplasia

OBJECTIVES: To determine the extent of observer agreement in diagnosis of oral epithelial dysplasia (OED). Published studies of OED examiner agreement report relatively low agreement levels; however, these studies were limited by the methodologies employed. METHODS: For this study, 64 slides were each independently examined twice by three oral pathologists. Consistency was assessed by determining intra- and interexaminer agreement. Conformity was assessed by using the modal diagnosis as a gold standard. RESULTS: The group showed moderate interobserver agreement when grading the presence or absence of OED with a group-simple kappa (Ks) of 0.51 (95% CI = 0.42-0.61), and substantial agreement when using a 5-point ordinal scale with a group-weighted kappa (Kw) of 0.74 (95% CI = 0.64-0.85). The group showed fair to substantial intraexaminer agreement when assessing the presence or absence of OED, with Ks ranging from 0.22 to 0.78, and showing almost a perfect agreement using a 5-point ordinal scale, with Kw ranging from 0.82-0.96. Conformity with the comparison standard modal diagnosis was almost perfect, with pairwise Kw ranging from 0.81 to 0.92. CONCLUSION: Overall, there was substantial intra- and interobserver consistency and almost perfect conformity in the grading of OED. Appropriate statistical methods are necessary to determine the degree of observer agreement.