Answers are grounded in this paper and cite their sources.
The observed degree of agreement between judges is commonly summarized using Cohen's (1960) kappa. Previous research has related values of kappa to the marginal distributions of the agreement matrix. This manuscript provides an approach for calculating maximum values of kappa as a function of observed agreement proportions between judges. Solutions are provided separately for matrices of size 2 x 2, 3 x 3, 4 x 4, and k X k; plots are provided for the 2 x 2, 3 x 3, and 4 x 4 matrices.
No takes yet. Share an insight, caveat, or question.
Umesh et al. (1989) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: