AI RESEARCH
Efficient Federated RLHF via Zeroth-Order Policy Optimization
arXiv CS.LG
•
ArXi:2604.17747v1 Announce Type: new This paper considers reinforcement learning from human feedback in a federated learning setting with resource-constrained agents, such as edge devices. We propose an efficient federated RLHF algorithm, named Partitioned, Sign-based Stochastic Zeroth-order Policy Optimization (Par-S$^2$ZPO). The algorithm is built on zeroth-order optimization with binary perturbation, resulting in low communication, computation, and memory complexity by design.