AI RESEARCH
OptEMA: Adaptive Exponential Moving Average for Stochastic Optimization with Zero-Noise Optimality
arXiv CS.LG
•
ArXi:2603.09923v1 Announce Type: new The Exponential Moving Average (EMA) is a cornerstone of widely used optimizers such as Adam. However, existing theoretical analyses of Adam-style methods have notable limitations: their guarantees can remain suboptimal in the zero-noise regime, rely on restrictive boundedness conditions (e.g., bounded gradients or objective gaps), use constant or open-loop stepsizes, or require prior knowledge of Lipschitz constants. To overcome these bottlenecks, we