AI RESEARCH
Demystifying Group Relative Policy Optimization: Its Policy Gradient is a U-Statistic
arXiv CS.LG
•
ArXi:2603.01162v3 Announce Type: replace Group relative policy optimization (GRPO), a core methodological component of DeepSeekMath and DeepSeek-R1, has emerged as a cornerstone for scaling reasoning capabilities of large language models. Despite its widespread adoption and the proliferation of follow-up works, the theoretical properties of GRPO remain less studied. This paper provides a unified framework to understand GRPO through the lens of classical U-statistics.