SOURCE-LINKED INTELLIGENCE

SGD in Multiclass Logistic Regression: Sequential Learning and Scaling Laws

arXiv · AI, language, vision and robotics · article · Sep 7, 2026 · UTC

We study the training dynamics of multiclass logistic regression on high-dimensional Gaussian mixture models with a large number of classes and establish precise scaling laws governing the cross-entropy risk under gradient-based optimization. We show that learning proceeds sequentially across classes, from most to least frequent. When the class priors follow a power law distribution, the risk dynamics decompose into three phases: an initial plateau until the first class is learned, a power-law decay regime during which sequential learning occurs, and a final convergence regime. We then analyze

Read original source ↗ Open in workspace

recordType

paper

region

Global