Entity · paper

Sharpness-Aware Minimization and Muon: Robustness under the Spectral Norm

paperactiveprovisionalsharpness-aware-minimization-and-muon-robustness-under-the-spectral-norm-a6c2293d·1 events·first seen 47h ago

Aliases: Sharpness-Aware Minimization and Muon: Robustness under the Spectral Norm

Co-occurring entities

Sharpness-Aware Minimization AdamW ImageNet Muon

More like this (12)

Sharpness-Aware Minimization Muon Optimizer Boundary Sharpening Invariant Risk Minimization Beyond Negative-Ridge Endpoints: Mixed-Sign Spectral Regularization via Negative-Shifted Gradient Descent Beyond Fully Random Masking: Attention-Guided Denoising and Optimization for Diffusion Language Models distributionally robust optimization quantization-induced degradation The Illusion of Equivalency: Statistical Characterization of Quantization Effects in LLMs Minimax Density Probabilistic Smoothing with Ratio-Monotone Transforms quantization-aware training

Recent events (1)

5arXiv · cs.LG·47h ago·source ↗

Spectral-norm SAM combined with Muon optimizer achieves strong ImageNet results

A new arXiv preprint introduces a matrix-aware variant of Sharpness-Aware Minimization (SAM) that uses a layerwise spectral inner perturbation for hidden-layer weights, combined with either AdamW/SGDW or the Muon optimizer for the outer update. Experiments on ImageNet-1K with ViT-Small/16 and ResNet-50 show the spectral SAM + Muon combination achieves the best validation accuracy among evaluated methods. The work connects the recently popular Muon optimizer's matrix-structure philosophy to the SAM generalization framework.

Training Infrastructure Evaluation and Benchmarking Sharpness-Aware Minimization AdamW ImageNet +2 more