Sample-Efficient Hypergradient Estimation for Decentralized Bi-Level Reinforcement Learning

ArXi:2603.14867v1 Announce Type: cross Many strategic decision-making problems, such as environment design for warehouse robots, can be naturally formulated as bi-level reinforcement learning (RL), where a leader agent optimizes its objective while a follower solves a Marko decision process (MDP) conditioned on the leader's decisions. In many situations, a fundamental challenge arises when the leader cannot intervene in the follower's optimization process; it can only observe the optimization outcome.