AI RESEARCH
Accelerating LMO-Based Optimization via Implicit Gradient Transport
arXiv CS.LG
•
ArXi:2605.05577v1 Announce Type: new Recent optimizers such as Lion and Muon have nstrated strong empirical performance by normalizing gradient momentum via linear minimization oracles (LMOs). While variance reduction has been explored to accelerate LMO-based methods, it typically incurs substantial computational overhead due to additional gradient evaluations. At the same time, the theoretical understanding of LMO-based methods remains fragmented across unconstrained and constrained formulations.