AI RESEARCH
Position: Zeroth-Order Optimization in Deep Learning Is Underexplored, Not Underpowered
arXiv CS.LG
•
ArXi:2605.15622v1 Announce Type: new Zeroth-order (ZO) optimization, learning from finite differences of function evaluations without backpropagation, has recently regained attention in deep learning due to its memory efficiency and applicability to gray- or black-box pipelines. Yet, ZO methods are often dismissed as fundamentally unscalable because of estimator variance and unfavorable query complexity. We argue that this