Answers are grounded in this paper and cite their sources.
Residual deep neural networks (ResNets) are mathematically described as interacting particle systems. In the case of infinitely many layers the ResNet leads to a system of coupled system of ordinary differential equations known as neural differential equations. For large scale input data we derive a mean–field limit and show well–posedness of the resulting description. Further, we analyze the existence of solutions to the training process by using both a controllability and an optimal control point of view. Numerical investigations based on the solution of a formal optimality system illustrate the theoretical findings.
No takes yet. Share an insight, caveat, or question.
Herty et al. (2022) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: