AI RESEARCH
From Cursed to Competitive: Closing the ZO-FO Gap via Input-to-State Stability
arXiv CS.LG
•
ArXi:2604.25372v1 Announce Type: cross While it is generally understood that zeroth-order (ZO) algorithms have an extra dependency on their number of iterations for any choice of parameters, compared to their first-order (FO) counterparts, in this work, we show that under several conditions, in expectation, ZO methods do not suffer from extra dimension dependencies in their convergence rates with respect to their FO counterparts.