On the Universal Approximation Property of Deep Fully Convolutional Neural Networks

.We study the approximation of shift-invariant or equivariant functions by deep fully convolutional networks from the dynamical systems perspective. We prove that deep residual fully convolutional networks and their continuous-layer counterparts can achieve universal approximation of these symmetric functions at constant channel width. Moreover, we show that the same can be achieved by nonresidual variants with at least two channels in each layer and convolutional kernel size of at least 2. In addition, we show that these requirements are necessary in the sense that networks with fewer channels or smaller kernels fail to be universal approximators.Keywordsapproximation theoryconvolutional networksdeep learninguniversal approximationinvarianceMSC codes68T0541A99