paper
Accelerated Decentralized Stochastic Gradient Descent for Strongly Convex Optimization
paperactiveprovisional
accelerated-decentralized-stochastic-gradient-descent-for-strongly-convex-optimization-88670200·1 events·first seen 9d agoAliases: Accelerated Decentralized Stochastic Gradient Descent for Strongly Convex Optimization
Co-occurring entities
More like this (12)
stochastic gradient ascentPrivate Stochastic Convex Optimizationdifferentiable convex optimizationrandomized coordinate descentGradient-Guided Reward Optimizationdistributed optimizationconjugate gradientDivergence Regularized Policy OptimizationGravity-Weighted Direct Preference OptimizationMulti-Gossip Accelerated DSGDHierarchical Advantage Weighting for Online RL Fine-Tuning of VLAs from Sparse Episode OutcomesSDE (Stochastic Differential Equation LR scaling)
Recent events (1)
MG-ADSGD achieves optimal communication complexity for decentralized stochastic strongly convex optimization
Researchers propose Multi-Gossip Accelerated DSGD (MG-ADSGD), a decentralized stochastic optimization algorithm that simultaneously achieves accelerated dependence on both the condition number (√κ) and the network spectral gap (1/√(1-β)), a combination no prior stochastic method had attained. The algorithm couples gossip depth with mini-batch size so that additional communication rounds improve both consensus accuracy and gradient variance reduction. The resulting communication complexity is claimed to be the best currently known for decentralized stochastic strongly convex optimization up to logarithmic factors.