AI RESEARCH
A Two-Timescale Primal-Dual Framework for Reinforcement Learning via Online Dual Variable Guidance
arXiv CS.LG
•
ArXi:2505.04494v3 Announce Type: replace-cross We study reinforcement learning by combining recent advances in regularized linear programming formulations with the classical theory of stochastic approximation. Motivated by the challenge of designing algorithms that leverage off-policy data while maintaining on-policy exploration, we propose PGDA-RL, a novel primal-dual Projected Gradient Descent-Ascent algorithm for solving regularized Marko Decision Processes (MDPs