Answers are grounded in this paper and cite their sources.
Reliability is a concept central to testing. Whatever measure of reliability is used, its value will always have to be estimated from a sample and will therefore be subject to sampling variation. The magnitude of sampling error involved in measuring reliability from a sample should be indicated by quoting the standard error of a reliability estimate or by giving a confidence interval for its true value. This paper considers several simple ANOVA models which can be used to define and estimate reliability coefficients. It discusses the statistical properties of commonly used measures of reliability and explains how to test hypotheses about their true values.
No takes yet. Share an insight, caveat, or question.
Krzanowski et al. (1984) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: