technique

KL-regularized RL

techniqueactiveprovisionalkl-regularized-rl-6685d9b1·1 events·first seen 16d ago

Aliases: KL-regularized RL

Co-occurring entities

Reinforcement Learning DRIFT importance-weighted supervised fine-tuning

More like this (12)

Entropy-Regularized Reinforcement Learning ExpRL: Exploratory RL for LLM Mid-Training KL-Cov regularization ExpRL Competitive Programming RL Constrained Reinforcement Learning Recursive Language Models (RLMs)RL² RLIF (Reinforcement Learning from Internal Feedback)TRL (Transformer Reinforcement Learning)L0 regularization Hierarchical Advantage Weighting for Online RL Fine-Tuning of VLAs from Sparse Episode Outcomes

Recent events (1)

6arXiv · cs.CL·16d ago·source ↗

DRIFT: Decoupled Rollouts and Importance-Weighted Fine-Tuning for Efficient Multi-Turn Optimization

DRIFT is a training framework that bridges online RL and offline SFT for multi-turn LLM optimization by exploiting the theoretical equivalence between KL-regularized RL and importance-weighted supervised learning. It decouples rollout generation from policy optimization: trajectories are sampled from a fixed reference policy offline, weighted by return-based importance scores, and used for weighted SFT. Empirically, DRIFT matches or exceeds multi-turn RL baselines while retaining the efficiency and simplicity of standard supervised fine-tuning. Code is publicly released.

Inference Economics Agent and Tool Ecosystem KL-regularized RL Reinforcement Learning DRIFT +2 more