Differentially Private Gradient-Tracking-Based Distributed Stochastic Optimization over Directed Graphs
원문 초록
This paper proposes a new differentially private gradient-tracking-based distributed stochastic optimization algorithm over directed graphs. Specifically, privacy noises are added to each agent's state and tracking variable to prevent information leakage, and then perturbed states and tracking variables are transmitted to neighbors. We design two novel schemes of the iteration step-sizes and the sampling number for the algorithm. By using the sampling parameter-controlled subsampling method, both schemes enhance the differential privacy level, and achieve the finite cumulative privacy budget even over infinite iterations. The convergence rate of the algorithm is shown for both nonconvex with the Polyak-Lojasiewicz condition and strongly convex objectives: Scheme (S1) achieves the polynomial convergence rate, and Scheme (S2) achieves the exponential convergence rate. The trade-off between the privacy and the convergence rate is presented. The algorithm's effectiveness and superior performance over the existing works are demonstrated through numerical examples of distributed training on benchmark datasets "MNIST" and "CIFAR-10".
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Adaptive Semantic Token Communication for Transformer-based Edge Inference
Efficient Optimization with Orthogonality Constraint: a Randomized Riemannian Submanifold Method
JaxSGMC: Modular stochastic gradient MCMC in JAX
SIM-Shapley: A Stable and Computationally Efficient Approach to Shapley Value Approximation
Adversarial Subspace Generation for Outlier Detection in High-Dimensional Data
CatCMA with Margin: Stochastic Optimization for Continuous, Integer, and Categorical Variables
토론
이 논문과 연결된 포럼 글이 아직 없습니다.