AI RESEARCH
Natural Hypergradient Descent: Algorithm Design, Convergence Analysis, and Parallel Implementation
arXiv CS.LG
•
ArXi:2602.10905v2 Announce Type: replace In this work, we propose Natural Hypergradient Descent (NHGD), a new method for solving bilevel optimization problems. To address the computational bottleneck in hypergradient estimation--namely, the need to compute or approximate Hessian inverse--we exploit the statistical structure of the inner optimization problem and use the empirical Fisher information matrix as an asymptotically consistent surrogate for the Hessian.