AI RESEARCH
Prune-OPD: Efficient and Reliable On-Policy Distillation for Long-Horizon Reasoning
arXiv CS.AI
•
ArXi:2605.07804v1 Announce Type: cross On-policy distillation (OPD) leverages dense teacher rewards to enhance reasoning models. However, scaling OPD to long-horizon tasks exposes a critical flaw: as the student's generated prefix inevitably diverges from the teacher's thought process, the teacher's dense reward loses local exploitability. Continuing to generate and evaluate tokens on these ``drifted'' trajectories not only degrades reward quality but also incurs massive computational waste. To address this, we