Revisiting LocalSGD and SCAFFOLD: Improved Rates and Missing Analysis
원문 초록
LocalSGD and SCAFFOLD are widely used methods in distributed stochastic optimization, with numerous applications in machine learning, large-scale data processing, and federated learning. However, rigorously establishing their theoretical advantages over simpler methods, such as minibatch SGD (MbSGD), has proven challenging, as existing analyses often rely on strong assumptions, unrealistic premises, or overly restrictive scenarios. In this work, we revisit the convergence properties of LocalSGD and SCAFFOLD under a variety of existing or weaker conditions, including gradient similarity, Hessian similarity, weak convexity, and Lipschitz continuity of the Hessian. Our analysis shows that (i) LocalSGD achieves faster convergence compared to MbSGD for weakly convex functions without requiring stronger gradient similarity assumptions; (ii) LocalSGD benefits significantly from higher-order similarity and smoothness; and (iii) SCAFFOLD demonstrates faster convergence than MbSGD for a broader class of non-quadratic functions. These theoretical insights provide a clearer understanding of the conditions under which LocalSGD and SCAFFOLD outperform MbSGD.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
FLsim: A Modular and Library-Agnostic Simulation Framework for Federated Learning
Domain Borders Are There to Be Crossed With Federated Few-Shot Adaptation
Geo-ORBIT: A Federated Digital Twin Framework for Scene-Adaptive Lane Geometry Detection
Prototype-Guided and Lightweight Adapters for Inherent Interpretation and Generalisation in Federated Learning
FedPall: Prototype-based Adversarial and Collaborative Learning for Federated Learning with Feature Drift
BackFed: An Efficient & Standardized Benchmark Suite for Backdoor Attacks in Federated Learning
토론
이 논문과 연결된 포럼 글이 아직 없습니다.