AI RESEARCH
Decoupling Reasoning and Confidence: Resurrecting Calibration in Reinforcement Learning from Verifiable Rewards
arXiv CS.LG
•
ArXi:2603.09117v1 Announce Type: new Reinforcement Learning from Verifiable Rewards (RLVR) significantly enhances large language models (LLMs) reasoning but severely suffers from calibration degeneration, where models become excessively over-confident in incorrect answers. Previous studies devote to directly incorporating calibration objective into existing optimization target. However, our theoretical analysis nstrates that there exists a fundamental gradient conflict between the optimization for maximizing policy accuracy and minimizing calibration error.