AI RESEARCH
Skip-Connected Policy Optimization for Implicit Advantage
arXiv CS.LG
•
ArXi:2604.08690v1 Announce Type: new Group Relative Policy Optimization (GRPO) has proven effective in RLVR by using outcome-based rewards. While fine-grained dense rewards can theoretically improve performance, we reveal that under practical sampling budgets, Monte Carlo estimation yields high-variance and sign-inconsistent advantages for early reasoning tokens, paradoxically underperforming outcome-only