return-to-go credit assignment

β-OPSD: Generalizing on-policy self-distillation via policy optimization equivalence

This paper identifies that vanilla on-policy self-distillation (OPSD) is a special case (β=1) of a broader policy-optimization family parameterized by a KL penalty weight β. The authors derive β-OPSD, whose optimal policy is a geometric interpolation between a reference policy and a privileged teacher, and implement it efficiently by mixing token-level logits rather than running full RL. Experiments on mathematical reasoning benchmarks show β-OPSD improves optimization stability and downstream performance over vanilla OPSD.