AI RESEARCH
Why Zeroth-Order Adaptation May Forget Less: A Randomized Shaping Theory
arXiv CS.LG
•
ArXi:2605.10658v1 Announce Type: new Continual learning requires new-task adaptation without damaging previously acquired capabilities. Recent forward-pass and zeroth-order (ZO) results show that low-query adaptation may retain better than first-order (FO) descent, but the usual view of ZO as noisy FO estimation does not explain why. We give a local randomized gradient-shaping analysis: finite differences expose a raw shape that is mean-aligned with FO, while the norm-matched comparator fixes the expected squared adaptation norm.