Key points are not available for this paper at this time.
We consider the problem of constructing a confidence interval for the intraclass correlation coefficient in an interrater reliability study when both raters and subjects are random effects in a two-way random effects model. A simulation study is conducted to investigate and compare the interval coverage per cents of three methods of approximation: (i) Satterthwaite's two moments, the standard approach; (ii) modified three moments, a newer approach; and (iii) modified large sample, the newest approach for this particular problem. One-sided lower and two-sided bounds are examined. For the two-moment approach, coverage of confidence intervals can be understated for not only one-sided bounds but also two-sided bounds. Coverage of confidence intervals for the modified large-sample approach is either correct or conservative and provides narrower two-sided widths than the modified three-moment approach, which can understate coverage on one-sided and two-sided bounds for two or three raters. The competing methods are illustrated with data from a reliability study of four raters evaluating the number of decayed, missing, and filled surfaces in the permanent teeth of ten subjects. Overall, the modified large-sample approach provides the most satisfactory performance of the three methods.
Cappelleri et al. (Tue,) studied this question.