AI RESEARCH
Fast Gauss-Newton for Multiclass Cross-Entropy
arXiv CS.LG
•
ArXi:2605.06081v1 Announce Type: new In multiclass softmax cross-entropy, the full generalized Gauss-Newton (GGN) curvature couples all output logits through the softmax covariance, making curvature-vector products harder to scale as the number of classes grows. We show that the standard multiclass GGN can be decomposed exactly into a true-vs-rest term and a positive semidefinite within-competitor covariance term.