technique

importance-weighted supervised fine-tuning

techniqueactiveprovisionalimportance-weighted-supervised-fine-tuning-aa29d384·1 events·first seen 16d ago

Aliases: importance-weighted supervised fine-tuning

Co-occurring entities

KL-regularized RL Reinforcement Learning DRIFT

More like this (12)

supervised fine-tuning Parameter-Efficient Fine-Tuning Hierarchical Advantage Weighting for Online RL Fine-Tuning of VLAs from Sparse Episode Outcomes reinforcement fine-tuning A Unifying Lens on Supervised Fine-Tuning Through Target Distribution Design Language Model Finetuning behavioral fine-tuning fine-tuning orthogonal finetuning weakly supervised anomaly detection Gravity-Weighted Direct Preference Optimization quantization-aware training

Recent events (1)

6arXiv · cs.CL·16d ago·source ↗

DRIFT: Decoupled Rollouts and Importance-Weighted Fine-Tuning for Efficient Multi-Turn Optimization

DRIFT is a training framework that bridges online RL and offline SFT for multi-turn LLM optimization by exploiting the theoretical equivalence between KL-regularized RL and importance-weighted supervised learning. It decouples rollout generation from policy optimization: trajectories are sampled from a fixed reference policy offline, weighted by return-based importance scores, and used for weighted SFT. Empirically, DRIFT matches or exceeds multi-turn RL baselines while retaining the efficiency and simplicity of standard supervised fine-tuning. Code is publicly released.

Inference Economics Agent and Tool Ecosystem KL-regularized RL Reinforcement Learning DRIFT +2 more