Answers are grounded in this paper and cite their sources.
We present a variational framework for studying functions learned by deep neural networks with rectified linear unit nonlinearities. We introduce a function space built from compositions of functions of second-order Radon-domain bounded variation. The compositional form of these functions captures the structure of deep neural networks. We prove a representer theorem that shows that deep neural networks with finite width solve regularized data-fitting problems over this space. The critical width is controlled by the square of the number of training data. This perspective explains the effect of weight-decay regularization in neural network training, the importance of skip connections, and the role of sparsity in neural networks. By considering the function-space perspective, we provide sharp links between deep learning and variational methods.
No takes yet. Share an insight, caveat, or question.
Parhi et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: