Artificial Intelligence Benchmarking of Emergency Department Triage Classifiers Under Unmeasured Nurse Acuity Re-Scoring at Queue Reordering: A Healthcare Stability Evaluation

Emergency department triage classifiers are benchmarked almost exclusively on static acuity labels, yet the operational label is produced when a nurse re-scores a patient at queue reordering. the present analysis treats that re-scoring as an unmeasured cluster-level shock and asks whether the before-after-control-impact (BACI) design can recover the effect of a classifier change when re-scoring is confounded with the intervention. A replication-based evaluation was conducted over a declared variance structure with between-cluster SD 6.0, residual SD 4.5 and intraclass correlation 0.64, using 2000 replications per design cell across 8, 16, 24 and 32 clusters and effects of 3, 5 and 8 percentage points. Power reached 80% at 52 clusters for a 5-percentage-point effect; at the largest design cell of 32 clusters the minimum detectable effect remained above the 5-point target. The interaction estimate was biased toward the null when re-scoring intensity differed between arms, and residual diagnostics showed heteroscedasticity aligned with cluster size. The contribution is a stability bound: BACI benchmarking of triage classifiers is defensible only when re-scoring is either measured or shown to be exchangeable across arms, and the paper specifies the design conditions under which that claim can be tested.