AI RESEARCH
AdaCubic: An Adaptive Cubic Regularization Optimizer for Deep Learning
arXiv CS.LG
•
ArXi:2604.09437v1 Announce Type: new A novel regularization technique, AdaCubic, is proposed that adapts the weight of the cubic term. The heart of AdaCubic is an auxiliary optimization problem with cubic constraints that dynamically adjusts the weight of the cubic term in Newton's cubic regularized method. We use Hutchinson's method to approximate the Hessian matrix, thereby reducing computational cost. We nstrate that AdaCubic inherits the cubically regularized Newton method's local convergence guarantees.