AI RESEARCH
Where Does Warm-Up Come From? Adaptive Scheduling for Norm-Constrained Optimizers
arXiv CS.LG
•
ArXi:2602.05813v2 Announce Type: replace We study adaptive learning rate scheduling for norm-constrained optimizers (e.g., Muon and Lion). We Building on this theory, we develop a practical learning rate scheduler that relies only on standard hyperparameters and adapts the warm-up duration automatically at the beginning of