SOURCE-LINKED INTELLIGENCE

Across the Loss Landscape with Progressive Growth

arXiv · AI, language, vision and robotics · article · Aug 25, 2026 · UTC

Deep neural networks generalize well despite their highly nonconvex, overparameterized loss landscapes, a phenomenon often associated with the geometry of the minima found by stochastic optimization. We study how incremental grow-and-optimize strategies bias training toward flatter regions by viewing growth as progressive constraint relaxation. Starting from a low-dimensional submodel, we iteratively expand the trainable parameters by unlocking nested random subspaces while freezing the orthogonal complement at the network initialization, re-optimizing after each expansion until the full archi

Read original source ↗ Open in workspace

recordType

paper

region

Global