Entity · technique

SLORR

techniqueactiveprovisionalslorr-3cd368f6·1 events·first seen 8h ago

Aliases: SLORR

Co-occurring entities

ViT-B/16 Hoyer sparsity ResNet-50 ImageNet

More like this (12)

TailLoR RLOO SLOG LoRA QLoRA Late-Stage LoRA RTLLM SHERLOC RL² SLS RLVR VRRL

Recent events (1)

4arXiv · cs.LG·8h ago·source ↗

SLORR: Stateless in-training low-rank regularization for compressible neural networks

Researchers introduce SLORR, a framework for in-training low-rank regularization that avoids SVD computation, architectural modifications, and stateful caching. Two variants based on Hoyer sparsity and nuclear norm are evaluated on ImageNet-1K (ResNet-50, ViT-B/16, ViT-L/16) and LLM pretraining at 135M and 560M parameter scales. SLORR-trained models compress more effectively than unregularized baselines while adding under 8% training overhead for vision models and under 1% for LLM pretraining, making post-training compression more viable.

Training Infrastructure Inference Economics ViT-B/16 SLORR Hoyer sparsity +2 more