A Diagnostic Trilemma for Residual Orthogonalization in Goodness‐of‐Fit Testing

Orthogonalized residuals identify model misspecification in sparse contingency tables and latent variable models. While many orthogonalization methods exist, the structural reasons for their different diagnostic behaviors remain unclear. This paper formalizes a diagnostic trilemma based on three desirable properties: sequential innovation orthogonality, localized interpretability and permutation invariance. We argue these objectives cannot generally be achieved simultaneously for correlated residuals. Triangular decompositions, including sequential sums of squares, QR, Cholesky and Helmert methods, maintain sequential innovation orthogonality and localized interpretability but lack permutation invariance. Spectral methods maintain permutation invariance but lack localized interpretability. Polar decompositions retain partial localization but fail to provide the sequential structures needed for innovation‐based diagnostics. The proposed framework clarifies the strengths and limitations of existing orthogonalization methods and helps explain why triangular decompositions are often preferred for directional diagnostics.