AI RESEARCH
Efficient Bilevel Optimization with KFAC-Based Hypergradients
arXiv CS.LG
•
ArXi:2603.29108v1 Announce Type: new Bilevel optimization (BO) is widely applicable to many machine learning problems. Scaling BO, however, requires repeatedly computing hypergradients, which involves solving inverse Hessian-vector products (IHVPs). In practice, these operations are often approximated using crude surrogates such as one-step gradient unrolling or identity/short Neumann expansions, which discard curvature information.