Key points are not available for this paper at this time.
Previously proposed methods for calculating the kappa measure of nominal rating agreement among multiple raters are not applicable in many situations. This paper presents a more general computational method which can be used across a broader range of rating designs, including those in which raters vary with respect to their base rates and pairs of raters vary with respect to how many subjects they rate in common. A Monte Carlo method for determining the statistical significance of this generalized kappa coefficient is discussed.
John S. Uebersax (Mon,) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: