SOURCE-LINKED INTELLIGENCE

Canalization Before Generalization: Grokking as a Dynamical Probe

arXiv · AI, language, vision and robotics · article · Aug 26, 2026 · UTC

For overparameterized neural networks, many solutions can fit the training data equally well while behaving very differently on unseen samples. Grokking separates training fit from visible generalization, providing a window for studying how this selection develops during training. We sweep short, fixed-duration weight-decay (WD) perturbations across the pre-generalization plateau and measure how they shift later generalization time. Across three grokking tasks, these shifts are unordered early in the plateau but later form a stable dose ordering, with stronger WD increases leading to earlier g

Read original source ↗ Open in workspace

recordType

paper

region

Global