Answers are grounded in this paper and cite their sources.
Jacot, Ged, Şimşek, Hongler and Gabriel described the small-initialization training of deep linear networks through the gradient flow of a homogeneous cost. This description needs the flow not to converge to a saddle, proved almost surely for depth at most three when the input and output dimensions agree, and with probability at least 1/2 for larger depth. Every critical point of the cost is a strict saddle or has vanishing Hessian. The conserved imbalances between layers almost surely exclude limits with two vanishing layers and, for hidden layers of equal width, limits with two vanishing products of adjacent layers, or a vanishing layer and such a product not containing it, unless both products contain one vanishing layer. This settles depth three for every gradient of full rank, and depth four when also the hidden layers have equal width. For larger depth we reduce the question to critical points with at most one vanishing layer and give a numerical example at depth eight that the conserved quantities cannot exclude.
No takes yet. Share an insight, caveat, or question.
Yitzchak Shmalo (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: