Integrating precision and power criteria in determining the number of raters needed for standard area diagram validation studies

Visual estimation of plant disease severity is widely used but inherently subjective, resulting in variability among raters. Standard area diagrams (SADs) are commonly employed to improve accuracy and precision; however, the number of raters required to validate SAD performance is often arbitrarily defined. This study aimed to establish an empirical framework to estimate the number of raters needed to achieve precise estimates of mean agreement and sufficient statistical power to detect improvements due to SAD use. Data from nine SAD validation studies across diverse plant disease pathosystems were analyzed. Agreement was quantified using Lin’s concordance correlation coefficient (LCCC). Two complementary criteria were applied: a precision criterion targeting a relative standard error of 0.10 for mean rater LCCC, and a power criterion evaluating absolute LCCC improvements of 0.05, 0.10, 0.15, and 0.20. Effects were assessed for feasibility relative to baseline agreement and converted exactly to the inverse hyperbolic tangent scale. The pooled precision estimate was seven raters, whereas study-specific precision estimates ranged from 2 to 16, indicating that the pooled value is not a universal recommendation. For an improvement of 0.10, combined sample-size estimates ranged from 5 to 32 raters across eight studies; this improvement was infeasible for pecan brown spot because baseline agreement was already 0.926. Required sample sizes depended on baseline agreement, variability in paired differences, and the specified improvement. These estimates provide context-specific guidance for SAD validation studies, although diagnostic and bootstrap analyses indicate that analytic power estimates require caution at small sample sizes. An interactive web application was developed to support adaptive estimation of rater numbers.