AI RESEARCH
ERPO: Token-Level Entropy-Regulated Policy Optimization for Large Reasoning Models
arXiv CS.AI
•
ArXi:2603.28204v1 Announce Type: cross Reinforcement learning from verifiable rewards (RLVR) has significantly advanced the reasoning capabilities of large language models. However, standard Group Relative Policy Optimization (GRPO) typically assigns a uniform, sequence-level advantage to all tokens, thereby overlooking the intrinsic information heterogeneity along reasoning chains. We show that this coarse-grained credit assignment leads to premature entropy collapse and encourages the model to generate redundant, low-quality reasoning paths.