AI RESEARCH
Regularized Centered Emphatic Temporal Difference Learning
arXiv CS.AI
•
ArXi:2605.04100v1 Announce Type: new Off-policy temporal-difference (TD) learning with function approximation faces a structural tradeoff among stability, projection geometry, and variance control. Emphatic TD (ETD) improves the off-policy projection geometry through follow-on emphasis, but the follow-on trace can have high variance. We revisit this tradeoff through Bellman-error centering. Although centering naturally removes a common drift term from TD errors, we show that a naive centered emphatic extension.