Implicit Bias of Mirror Flow in Homogeneous Neural Networks: Sparse and Dense Feature Learning

ArXi:2605.19458v1 Announce Type: new We study the max-margin solutions reached by mirror flow in deep neural networks with homogeneous activation functions. Extending classical results on gradient flow, we derive a novel balance equation for mirror flow from convex duality, enabling a characterization of the horizon function governing the induced margin. We further establish max-margin characterizations together with convergence rates and norm growth estimates. Finally, we our theory through experiments on synthetic datasets and standard vision tasks.