Offline Constrained Reinforcement Learning under Partial Data Coverage

ArXi:2505.17506v2 Announce Type: replace-cross We study offline constrained reinforcement learning with general function approximation in discounted constrained Marko decision processes. Prior methods either require full data coverage for evaluating intermediate policies, lack oracle efficiency, or requires the knowledge of data-generating distribution for policy extraction. We propose PDOCRL, an oracle-efficient primal-dual algorithm based on a decomposed linear-programming formulation that makes the policy an explicit optimization variable.