paper
TailLoR: Protecting Principal Components in Parameter-Efficient Continual Learning
paperactiveprovisional
taillor-protecting-principal-components-in-parameter-efficient-continual-learning-1dbc2fe8·1 events·first seen 12d agoAliases: TailLoR: Protecting Principal Components in Parameter-Efficient Continual Learning
Co-occurring entities
More like this (12)
Sparse Subspace-to-Expert Sharing for Task-Agnostic Continual LearningThe Stable Recovery Manifold: Geometric Principles Governing Recoverability in Continual LearningContinual LLM Upcycling: A Predictor-Gated Bank-Wise Sparsity Training Recipe for Dense-to-Sparse LLMsPC Layer: Polynomial Weight Preconditioning for Improving LLM Pre-TrainingRange Penalization: Theoretical Insights with Applications in Federated LearningFlashbackCL: Mitigating Temporal Forgetting in Federated LearningKL-regularized RLLearning from the Self-future: On-policy Self-distillation for dLLMsOperator LearningExpRL: Exploratory RL for LLM Mid-TrainingLanguage Modeling LossPreserving Plasticity in Continual Learning via Dynamical Isometry
Recent events (1)
TailLoR: Spectral-domain continual learning via protected principal components
TailLoR is a new parameter-efficient finetuning method for continual learning that uses the singular value decomposition of pre-trained weights as a fixed reference frame, applying low-rank updates only to the singular value matrix. A soft spectral penalty discourages updates aligned with dominant singular directions, reducing catastrophic interference while routing adaptation into long-tail spectral coordinates. The approach targets the forgetting problem in continual learning through a principled spectral lens.