Pure_Identity_Residual_Learning__Towards_a_Normalization_Free_Paradigm

Normalization layers have long been regarded as an indispensable component for stabilizing thetraining of deep residual networks. While recent studies have attempted to remove normalization, aunified theoretical framework to explain the underlying mechanism is still lacking. Built upon ourprior work on Zero-Gate Residual Networks (ZGRN), we formalize the Pure Identity ResidualLearning (PIRL) principle: by enforcing strict identity mapping on the main shortcut path andinitializing the residual branch to zero via learnable gating, deep neural networks can be trainedstably without any normalization layers.We provide rigorous mathematical analysis to prove that PIRL guarantees exact identity propagation at initialization and creates structural protection against gradient attenuation. Although weempirically validate PIRL on convolutional neural networks (CNNs), this principle is independentof specific network operators and has broad theoretical generalizability. We further propose a set oftestable hypotheses to demonstrate how PIRL can resolve long-standing bottlenecks in Transformerbased models, scientific computing and edge AI deployment.This paper is presented as a position paper. We do not claim universal empirical validationacross all domains. Instead, we put forward a theoretically sound new paradigm and a clear researchroadmap, inviting the research community to further explore the potential of normalization-free deeplearning.