Answers are grounded in this paper and cite their sources.
The function space of deep-learning machines is investigated by studying growth in the entropy of functions of a given error with respect to a reference function, realized by a deep-learning machine. Using physics-inspired methods we study both sparsely and densely connected architectures to discover a layerwise convergence of candidate functions, marked by a corresponding reduction in entropy when approaching the reference function, gain insight into the importance of having a large number of layers, and observe phase transitions as the error increases.
No takes yet. Share an insight, caveat, or question.
Li et al. (2018) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: