Practical Differences Among Aggregate-Level Conditional Status Metrics

Aggregate-level conditional status metrics (ACSMs) describe the status of a group by referencing current performance to expectations given past scores. This article provides a framework for these metrics, classifying them by aggregation function (mean or median), regression approach (linear mean and nonlinear quantile), and the scale that supports interpretations (percentile rank and score scale), among other factors. This study addresses the question “how different are these ACSMs?” in three ways. First, using simulated data, it evaluates how well each model recovers its respective parameters. Second, using both simulated and empirical data, it illustrates practical differences among ACSMs in terms of pairwise rank differences incurred by switching between metrics. Third, it ranks ACSMs in terms of their robustness under scale transformations. The results consistently show that choices between mean- and median-based metrics lead to more substantial differences than choices between fixed- and random-effects or linear mean and nonlinear quantile regression. The findings set expectations for cross-metric comparability in realistic data scenarios.