SOURCE-LINKED INTELLIGENCE

Geometry of learning dynamics: Gradient descent versus natural gradient on the ridge of optimization

arXiv · AI, language, vision and robotics · article · Sep 15, 2026 · UTC

High-capacity associative memories based on Kernel Logistic Regression (KLR) exhibit a "Ridge of Optimization" characterized by extreme stability and a highly skewed weight spectrum. However, the dynamical process by which learning converges to this critical regime has remained unclear. This paper provides a geometric analysis of the learning trajectories on the statistical manifold of a KLR-trained Hopfield network. By comparing the paths of Gradient Descent (GD) and Natural Gradient Descent (NGD), we elucidate the mechanisms governing the optimization process. Our analysis reveals that learn

Read original source ↗ Open in workspace

recordType

paper

region

Global