MAR loss

Theoretical analysis shows MIM more robust than contrastive learning in distributed self-supervised learning under non-IID data

A new arXiv preprint provides a rigorous theoretical analysis of distributed self-supervised learning (D-SSL) frameworks under non-IID (heterogeneous) data conditions. The key findings are that Masked Image Modeling (MIM) is inherently more robust to data heterogeneity than Contrastive Learning (CL), and that federated learning is no less robust than fully decentralized learning due to network connectivity effects. The authors also introduce MAR loss, a refinement of the MIM objective with local-to-global alignment regularization, validated across multiple architectures and distributed settings.