Key points are not available for this paper at this time.
Published formulas for the large sample variance of the kappa statistic that are appropriate for the case of different sets of raters for different Ss, when each set of raters is selected at random from a larger pool of available raters, are determined to be incorrect. New formulas are derived and c
Fleiss et al. (1979) studied this question.