paper
Sparse Subspace-to-Expert Sharing for Task-Agnostic Continual Learning
paperactiveprovisional
sparse-subspace-to-expert-sharing-for-task-agnostic-continual-learning-2726e327·1 events·first seen 9d agoAliases: Sparse Subspace-to-Expert Sharing for Task-Agnostic Continual Learning
Co-occurring entities
More like this (12)
Sparse Mixture-of-ExpertsMulti-Task LearningUnsupervised Continual Clustering via Forward-Backward Knowledge DistillationTailLoR: Protecting Principal Components in Parameter-Efficient Continual LearningKSAA-2026 Shared TaskUnstable Features, Reproducible Subspaces: Understanding Seed Dependence in Sparse AutoencodersRank-Constrained Subspace Learning (RCSL)Hierarchical Advantage Weighting for Online RL Fine-Tuning of VLAs from Sparse Episode OutcomesThe Stable Recovery Manifold: Geometric Principles Governing Recoverability in Continual LearningFlashbackCL: Mitigating Temporal Forgetting in Federated LearningLayer-Adaptive Expert PruningAlternating Token-Weighted Unlearning
Recent events (1)
SETA: Sparse Subspace-to-Expert Sharing for Continual Learning in LLMs
Researchers introduce SETA (Mixture of Sparse Experts for Task Agnostic Continual Learning), a framework addressing catastrophic forgetting in LLMs via adaptive sparse subspace decomposition into task-specific and shared expert modules. The approach uses adaptive elastic anchoring and routing-aware regularization to protect shared knowledge at both weight and routing levels. Experiments on LLaMA-2 7B and Qwen3-4B show competitive or superior performance versus continual learning baselines, with strong retention of early-task knowledge.