AI RESEARCH
Adapt or Forget: Provable Tradeoffs Between Adam and SGD in Nonstationary Optimization
arXiv CS.LG
•
ArXi:2605.04269v1 Announce Type: cross We provide a theoretical analysis of Adam under non-stationary stochastic objectives, separating two regimes: Euclidean tracking under adaptive strong monotonicity of the Adam-preconditioned mean-gradient operator, and high-probability projected stationarity guarantees under general $L$-smooth objectives.