AI RESEARCH
Optimize Wider, Not Deeper: Consensus Aggregation for Policy Optimization
arXiv CS.AI
•
ArXi:2603.12596v1 Announce Type: cross Proximal policy optimization (PPO) approximates the trust region update using multiple epochs of clipped SGD. Each epoch may drift further from the natural gradient direction, creating path-dependent noise. To understand this drift, we can use Fisher information geometry to decompose policy updates into signal (the natural gradient projection) and waste (the Fisher-orthogonal residual that consumes trust region budget without first-order surrogate improvement.