Bellman Residual Minimization for Control: Geometry, Stationarity, and Convergence

ArXi:2601.18840v3 Announce Type: replace Marko decision problems are most commonly solved via dynamic programming. Another approach is Bellman residual minimization, which directly minimizes the squared Bellman residual objective function. However, compared to dynamic programming, this approach has received relatively less attention, mainly because it is often less efficient in practice and can be difficult to extend to model-free settings such as reinforcement learning.