AI RESEARCH
Stabilizing Rubric Integration Training via Decoupled Advantage Normalization
arXiv CS.AI
•
ArXi:2603.26535v1 Announce Type: new We propose Process-Aware Policy Optimization (PAPO), a method that integrates process-level evaluation into Group Relative Policy Optimization (GRPO) through decoupled advantage normalization, to address two limitations of existing reward designs. Outcome reward models (ORM) evaluate only final-answer correctness, treating all correct responses identically regardless of reasoning quality, and gradually lose the advantage signal as groups become uniformly correct.