SOURCE-LINKED INTELLIGENCE

Revisiting Distributed Sign-Based Variance Reduction

arXiv · AI, language, vision and robotics · article · Sep 16, 2026 · UTC

Sign-based methods reduce communication costs in distributed environments, but aggregating local signs can introduce bias when data are heterogeneous. As a result, existing sign-based variance reduction methods fail to obtain the optimal convergence rates. In this paper, we solve this problem and obtain optimal rates for both nonconvex stochastic and finite-sum optimization. We first give a counterexample showing that majority voting can fail to approach stationary points even with exact local gradients. Motivated by this limitation, we propose tracking the global gradient at the server throug

Read original source ↗ Open in workspace

recordType

paper

region

Global