technique

Alternating Token-Weighted Unlearning

techniqueactiveprovisionalalternating-token-weighted-unlearning-a17257f8·1 events·first seen 12d ago

Aliases: Alternating Token-Weighted Unlearning

Co-occurring entities

More like this (12)

Backdoor Unlearning Generalization: A Path Toward the Removal of Unknown Triggers in LLMs inference-time behavioural unlearning Hierarchical Advantage Weighting for Online RL Fine-Tuning of VLAs from Sparse Episode Outcomes Entropy-Regularized Reinforcement Learning Operator Learning Language Modeling Loss Sparse Subspace-to-Expert Sharing for Task-Agnostic Continual Learning CLP: Collocation-Length Prediction for Zero-Loss Adaptive Multi-Token Inference KL-regularized RL Class-Incremental Learning Multi-Task Learning masked-token modeling

Recent events (1)

5arXiv · cs.CL·12d ago·source ↗

ATWU: Token-level importance learning improves LLM unlearning via retain-conflict criterion

This paper introduces Alternating Token-Weighted Unlearning (ATWU), a framework that learns which tokens in a forget sample are most relevant to unlearning by characterizing their conflict with the retain objective. Rather than relying on auxiliary models or heuristics, ATWU jointly learns token forget-specificity and model parameters using a lightweight linear scorer over hidden states. Evaluated on TOFU and RWKU benchmarks, ATWU achieves state-of-the-art forget-retain trade-offs and produces token-level scores that align with ground-truth forget-specific spans.

Evaluation and Benchmarking AI Safety Research RWKU Alternating Token-Weighted Unlearning TOFU