AI RESEARCH
On-Policy Context Distillation for Language Models
arXiv CS.CL
•
ArXi:2602.12275v2 Announce Type: replace Context distillation enables language models to internalize in-context knowledge into their parameters. In our work, we propose On-Policy Context Distillation (OPCD), a framework that bridges on-policy distillation with context distillation by