FlowAdam: Implicit Regularization via Geometry-Aware Soft Momentum Injection

ArXi:2604.06652v1 Announce Type: new Adaptive moment methods such as Adam use a diagonal, coordinate-wise preconditioner based on exponential moving averages of squared gradients. This diagonal scaling is coordinate-system dependent and can struggle with dense or rotated parameter couplings, including those in matrix factorization, tensor decomposition, and graph neural networks, because it treats each parameter independently. We