technique

on-policy self-distillation

techniqueactiveon-policy-self-distillation-6b9c43a2·2 events·first seen 29d ago

Aliases: on-policy self-distillation, Self-Policy Distillation (SPD)

Co-occurring entities

large language models key-value (KV) activation projection low-rank subspace projection Multimodal Large Language Models Thinking-with-Images Vision-OPD regional-to-global perception gap

More like this (12)

on-policy distillation On-Policy Co-Distillation On-Policy Distillation (OPD)Learning from the Self-future: On-policy Self-distillation for dLLMs Self-Distillation Dense Supervision, Sparse Updates: On the Sparsity and Geometry of On-Policy Distillation Canonical-Context On-Policy Distillation (CCOPD)ZEDA (Zero-Expert Self-Distillation Adaptation)Skill-Conditioned Gated Self-Distillation (SGSD)knowledge distillation Model Distillation self-play reinforcement learning

Recent events (2)

6arXiv · cs.CL·25d ago·source ↗

Self-Policy Distillation via Capability-Selective Subspace Projection

This paper introduces Self-Policy Distillation (SPD), a self-distillation method for LLMs that requires no external signals such as correctness filters or reward models. SPD extracts a low-rank capability subspace from the model's own gradients on correctness-defining tokens, then projects KV activations into this subspace during self-generation to isolate task-relevant signal from stylistic noise. Experiments across code generation, math reasoning, and QA show up to 13% improvement over prior signal-free self-distillation methods and 15% better out-of-domain generalization.

Frontier Model Releases Evaluation and Benchmarking large language models key-value (KV) activation projection low-rank subspace projection +2 more

6arXiv · cs.CL·29d ago·source ↗

Vision-OPD: On-Policy Self-Distillation for Fine-Grained Visual Understanding in MLLMs

Vision-OPD addresses a 'regional-to-global perception gap' in multimodal LLMs, where models answer fine-grained visual questions more accurately when given cropped evidence regions than full images. The method instantiates a crop-conditioned teacher and full-image-conditioned student from the same MLLM, minimizing token-level divergence along on-policy rollouts to transfer regional perception to the full-image policy. This self-distillation requires no external teacher models, ground-truth labels, reward verifiers, or inference-time tools. Benchmarks show competitive or superior performance against larger open-source, closed-source, and agentic 'Thinking-with-Images' models.

Evaluation and Benchmarking Agent and Tool Ecosystem Multimodal Large Language Models Thinking-with-Images on-policy self-distillation +4 more