What type of study is this?

This is a Quantitative Study study.

synapse

⌘+K

synapse

⌘+K

October 19, 2025Open Access

Finite-Dimensional Gaussian Approximation for Deep Neural Networks: Universality in Random Weights

Key Points

Gaussian approximation bounds in Wasserstein-1 norm established for deep neural networks with random weights.
Convergence rates of order n^{-({1}/{6})^{L-1} + ε} obtained for layer widths proportional to a common scale parameter n.
The study incorporates assumptions of Lipschitz activation functions and allows layer widths to grow infinitely.
Findings imply potential applications of Gaussian approximations in understanding deep learning behavior.

Abstract

We study the Finite-Dimensional Distributions (FDDs) of deep neural networks with randomly initialized weights that have finite-order moments. Specifically, we establish Gaussian approximation bounds in the Wasserstein-1 norm between the FDDs and their Gaussian limit assuming a Lipschitz activation function and allowing the layer widths to grow to infinity at arbitrary relative rates. In the special case where all widths are proportional to a common scale parameter n and there are L-1 hidden layers, we obtain convergence rates of order n^- ({1/6) ^L-1 + ε}, for any ε> 0.

Read Full Paperexternally

Mark Helpful

Bookmark

Relay

View Full Paper