Depth, Not Data: An Analysis of Hessian Spectral Bifurcation

ArXi:2602.00545v2 Announce Type: replace The eigenvalue distribution of the Hessian matrix plays a crucial role in understanding the optimization landscape of deep neural networks. Prior work has attributed the well-documented ``bulk-and-spike'' spectral structure, where a few dominant eigenvalues are separated from a bulk of smaller ones, to the imbalance in the data covariance matrix. In this work, we challenge this view by nstrating that such spectral Bifurcation can arise purely from the network architecture, independent of data imbalance.