AI RESEARCH
Weak-to-Strong Elicitation via Mismatched Wrong Drafts
arXiv CS.LG
•
ArXi:2605.17314v1 Announce Type: cross We consider whether off-policy experience from a smaller, weaker model can elicit capability in a stronger learner that on-policy RL fine-tuning (e.g., GRPO) does not reach. We find that injecting mathematically wrong drafts from a smaller but domain-trained model -- mismatched to the current problem -- into a stronger learner's GRPO context consistently outperforms standard on-policy GRPO on held-out MATH-500 and out-of-distribution AIME 2025/2026.