AI RESEARCH
Teacher-Guided Policy Optimization for LLM Distillation
arXiv CS.AI
•
ArXi:2605.13230v1 Announce Type: cross The convergence of reinforcement learning and imitation learning has positioned Reverse KL (RKL) as a promising paradigm for on-policy LLM distillation, aiming to unify exploration with teacher supervision. However, we identify a critical limitation: when the student and teacher distributions diverge significantly, standard RKL often fails to yield meaningful improvement due to uninformative negative feedback.