AI RESEARCH
DGPO: Distribution Guided Policy Optimization for Fine Grained Credit Assignment
arXiv CS.LG
•
ArXi:2605.03327v1 Announce Type: new Reinforcement learning is crucial for aligning large language models to perform complex reasoning tasks. However, current algorithms such as Group Relative Policy Optimization suffer from coarse grained, sequence level credit assignment, which severely struggles to isolate pivotal reasoning steps within long Chain of Thought generations. Furthermore, the standard unbounded Kullback Leibler divergence penalty induces severe gradient instability and mode seeking conservatism, ultimately stifling the discovery of novel reasoning trajectories.