Error Injection in Task-Based Image Quality Pipelines: What Regression Testing Cannot Catch, and Why Neither Internal Identities nor Closed-Form References Suffice Alone

Task-based image quality assessment—the modulation transfer function, the noise power spectrum (NPS), the noise-equivalent quanta and model observers—fails by returning a plausible wrong number rather than an error, and the regression test most implementations carry cannot tell a plausible right answer from a plausible wrong one, because the stored reference was recorded from the defective code. We injected six defects into a validated implementation of that chain through a severity dial that recovers the correct pipeline exactly at zero. A self-consistency regression test detected none of the six. Four internal identities, which need no ground truth, and two closed-form references, which need a phantom whose answer is known, together detected all six, in every case at or before the severity at which the reported detectability index d′ became wrong by more than 5%—an error that three of the six never produced. Neither family sufficed alone: the internal identities detected three of the six and the closed-form references five, and peak errors in a reported d′ reached 90%. Run unmodified on measured American College of Radiology (ACR) phantom projections across seven reconstruction kernels, the three identities that can be evaluated without ground truth transferred intact—Parseval held to 4×10−16—but their tolerances did not, and the strongest apodisation drove the noise dynamic range to within 0.6% of the threshold beyond which a prewhitening observer should return no value at all rather than a computed one, because 1/NPS has ceased to be numerically meaningful.