SOURCE-LINKED INTELLIGENCE

Gradient Descent with Stochastic Subspaces via Persistence of Memory

arXiv · AI, language, vision and robotics · article · Sep 16, 2026 · UTC

Stochastic subspace methods have gained popularity as gradient descent based techniques for large scale optimisation problems, especially in distributed settings. In this paper, we introduce the technique of "persistence of memory" to greatly extend and improve the random subspace methods. To this end, we leverage a vector that is only weakly correlated with the gradient in order to provide a guiding structure to the generative process of the random subspace along which the descent is going to take place. This guidance vector may be fixed for a large number of iterations, only to be refreshed

Read original source ↗ Open in workspace

recordType

paper

region

Global