AI RESEARCH
LionMuon: Alternating Spectral and Sign Descent for Efficient Training
arXiv CS.LG
•
ArXi:2605.19811v1 Announce Type: new In large-scale optimization, the cheapness and effectiveness of update steps are the most crucial factors for a successful optimizer. Sign-based optimizers like Lion or Signum produce cheap per-step updates, whereas Muon's spectral matrix-date gives a much stronger direction at a substantially higher per-step cost. In this work, we propose LionMuon, which retains the effectiveness of Muon steps while considerably cutting the averaged iteration cost, similar to sign-based methods.