The Physics of Multi-Turn Long-Horizon Planning: From Pre-training to Post-training via Single- and Multi-Teacher On-Policy Agentic Distillation

Unified study of multi-turn long-horizon planning across pre-training and post-training stages via agentic distillation

A new arXiv preprint introduces a controlled multi-turn environment to systematically study how long-horizon planning ability is acquired, shaped, and integrated in foundation model agents across three stages: pre-training data design, post-training via GRPO and on-policy distillation (OPD), and multi-teacher on-policy distillation (MOPD). Key findings include that explicit world model construction via chain-of-thought state transition modeling improves generalization, suboptimal trajectories severely degrade performance over long horizons, and OPD outperforms GRPO in low-quality and long-horizon settings. The multi-teacher distillation analysis reveals that compatible planning patterns enable cross-environment generalization while conflicting patterns cause interference.