IRIS: Interpolative R\'enyi Iterative Self-play for Large Language Model Fine-Tuning

ArXi:2604.20933v1 Announce Type: new Self-play fine-tuning enables large language models to improve beyond supervised fine-tuning without additional human annotations by contrasting annotated responses with self-generated ones. Many existing methods rely on a fixed divergence regime. SPIN is closely related to a KL-based regime, SPACE to a Jensen-Shannon-style objective via noise contrastive estimation, and SPIF to $\chi^2$-regularized self-play.