technique
Multi-Gossip Accelerated DSGD
techniqueactiveprovisional
multi-gossip-accelerated-dsgd-adb67da1·1 events·first seen 9d agoAliases: Multi-Gossip Accelerated DSGD
Co-occurring entities
More like this (12)
Gibbs-Accelerated Discrete Diffusion (GADD)SGSDAccelerated Decentralized Stochastic Gradient Descent for Strongly Convex OptimizationDPG BenchmarkGSPO (Group Sequence Policy Optimization)PyTorch DDPDirected Interaction GraphContext-Driven Incremental Compression for Multi-Turn Dialogue GenerationContext-Driven Incremental Compression for Multi-Turn Dialogue GenerationDual Layer Aggregation (DLA)DGX SparknDCG@10
Recent events (1)
MG-ADSGD achieves optimal communication complexity for decentralized stochastic strongly convex optimization
Researchers propose Multi-Gossip Accelerated DSGD (MG-ADSGD), a decentralized stochastic optimization algorithm that simultaneously achieves accelerated dependence on both the condition number (√κ) and the network spectral gap (1/√(1-β)), a combination no prior stochastic method had attained. The algorithm couples gossip depth with mini-batch size so that additional communication rounds improve both consensus accuracy and gradient variance reduction. The resulting communication complexity is claimed to be the best currently known for decentralized stochastic strongly convex optimization up to logarithmic factors.