Entity · technique

On-Policy Co-Distillation

techniqueactiveon-policy-co-distillation-82ee8218·1 events·first seen Jun 15, 2026

Aliases: On-Policy Co-Distillation

Co-occurring entities

Be My Tutor: On-Policy Co-Distillation for Mutual LLM Improvement via Peer Feedback

More like this (12)

On-Policy Distillation (OPD)on-policy distillation Routing-based On-Policy Distillation on-policy self-distillation Pass the Baton: Trajectory-Relayed On-Policy Distillation Multi-Teacher On-Policy Distillation Canonical-Context On-Policy Distillation (CCOPD)On-Policy Self-Distillation with Sampled Demonstrations Reduces Output Diversity Weak-to-Strong Generalization via Direct On-Policy Distillation Be My Tutor: On-Policy Co-Distillation for Mutual LLM Improvement via Peer Feedback Purified OPSD: On-Policy Self-Distillation Without Losing How to Think On-Policy Distillation for LLM Safety: A Routing Approach to Template-Robust Realignment

Recent events (1)

5arXiv · cs.CL·Jun 15, 2026·source ↗

OPCoD: On-Policy Co-Distillation for Mutual LLM Improvement via Peer Feedback

Researchers introduce On-Policy Co-Distillation (OPCoD), a training framework where two LLMs, each stronger in a different domain, iteratively tutor each other using on-policy rollouts and peer feedback. The method uses cognizance-based gating to control when feedback is given and feedback anchoring to ground it in the problem context. On Science Q&A tasks, OPCoD achieves Pareto improvement for both models across all evaluated domain pairs, outperforming one-way distillation and single-model fine-tuning baselines.

Evaluation and Benchmarking Alignment and RLHF On-Policy Co-Distillation Be My Tutor: On-Policy Co-Distillation for Mutual LLM Improvement via Peer Feedback