AI RESEARCH
Understanding and Preventing Entropy Collapse in RLVR with On-Policy Entropy Flow Optimization
arXiv CS.AI
•
ArXi:2605.11491v1 Announce Type: cross Reinforcement learning with verifiable rewards (RLVR) has become an effective paradigm for improving the reasoning ability of large language models. However, widely used RLVR algorithms, such as GRPO, often suffer from entropy collapse, leading to premature determinism and unstable optimization. Existing remedies, including entropy regularization and ratio-based clipping heuristics, either control entropy in a coarse-grained manner or rely on approximate on-policy