AI RESEARCH
Embarrassingly Simple Self-Distillation Improves Code Generation
arXiv CS.CL
•
ArXi:2604.01193v1 Announce Type: new Can a large language model (LLM) improve at code generation using only its own raw outputs, without a verifier, a teacher model, or reinforcement learning? We answer in the affirmative with simple self-distillation (SSD): sample solutions from the model with certain temperature and truncation configurations, then fine-tune on those samples with standard supervised fine-tuning.