SOURCE-LINKED INTELLIGENCE

How Edge of Stability Hinders SCAFFOLD in Federated Optimization

arXiv · AI, language, vision and robotics · article · Aug 26, 2026 · UTC

In federated learning, it is well known that heterogeneous data can (in theory) slow down optimization, and much effort has been directed at designing optimization algorithms that are unaffected by data heterogeneity, such as the SCAFFOLD algorithm. Yet, despite strong theoretical guarantees, SCAFFOLD does not usually outperform the much simpler FedAvg in practice. In this work, we propose that this gap is due to the presence of Edge of Stability (EoS) and progressive sharpening in federated optimization, supported by extensive empirical probing. First, we find that EoS-like dynamics occur wit

Read original source ↗ Open in workspace

recordType

paper

region

Global