AI RESEARCH
Delightful Gradients Accelerate Corner Escape
arXiv CS.LG
•
ArXi:2605.11908v1 Announce Type: new Softmax policy gradient converges at $O(1/t)$, but its transient behavior near sub-optimal corners of the simplex can be exponentially slow. The bottleneck is self-trapping: negative-advantage actions reinforce the corner policy and can initially push the optimal action backward. We study \emph{Delightful Policy Gradient} (DG), which gates each policy-gradient term by the product of advantage and action surprisal.