SOURCE-LINKED INTELLIGENCE

A Theoretical Analysis of Generalization Dynamics in Neural Networks under Gradient Descent with Weight Decay

arXiv · AI, language, vision and robotics · article · Sep 7, 2026 · UTC

Understanding generalization remains a central challenge in machine learning because it requires jointly considering data, architecture, and training dynamics. In this paper, we develop a theoretical framework that characterizes how these factors jointly shape generalization performance throughout training. More precisely, we study a broad class of neural networks trained under the $\ell^2$ loss by gradient descent (GD) with weight decay, and prove the convergence of GD to a neighbourhood of the global minimizers of the empirical loss. By partitioning the space based on the input data, we then

Read original source ↗ Open in workspace

recordType

paper

region

Global