SOURCE-LINKED INTELLIGENCE

EGGROLL, Unrolled: Understanding and Improving Low-Rank Evolution Strategies at Scale

arXiv · AI, language, vision and robotics · article · Sep 10, 2026 · UTC

EGGROLL makes evolution strategies (ES) practical for LLMs by replacing dense Gaussian weight perturbations with low-rank Gaussian products, often of rank one. This choice is computationally attractive but geometrically severe: each rank-one perturbation lies in a zero-volume subset of the ambient matrix space, despite having identity covariance. We characterize the mean EGGROLL update field at finite rank and nonzero perturbation radii, then analyze the error of its finite-population estimator. The population field is obtained by applying an explicit resolvent to the gradient of the objective

Read original source ↗ Open in workspace

recordType

paper

region

Global