AI RESEARCH
Optimizer-Induced Mode Connectivity: From AdamW to Muon
arXiv CS.AI
•
ArXi:2605.09991v1 Announce Type: new Mode connectivity has been widely studied, yet the role of the optimizer remains underexplored. We revisit it through optimizer-induced implicit regularization, asking how connectivity behaves when restricted to solutions constrained by a given optimizer. For two-layer ReLU networks, we show that solutions from a single optimizer -- AdamW, Muon, or others in the Lion-$\mathcal{K}$ family -- form a connected set at sufficiently large width, a result not implied by prior work.