This work studies how overparameterization reshapes the loss landscape of one-hidden-layer ReLU networks. On the theory side, it proves that for convex \ (L\) -Lipschitz losses with \ (₁\) -regularization on the output layer, any two models at the same loss level can be connected by a continuous path with arbitrarily small excess loss \ (\), extending earlier quadratic-loss connectivity results to a broader class of objectives (including logistic/cross-entropy settings). It also derives an asymptotic bound on the energy barrier, \ (= O (m^-) \), showing that the barrier vanishes as width \ (m\) increases, so sublevel sets become connected in the infinite-width limit. Empirically, Dynamic String Sampling experiments on synthetic Moons data and the Wisconsin Breast Cancer dataset show smaller pairwise barriers for wider networks; a permutation test on the maximum gap gives \ (p₄ₑ₌=0\), indicating a clear reduction in worst-case barrier height with width.
Saveliy Baturin (Wed,) studied this question.