Gaussian Equivalence for Self-Attention: Asymptotic Spectral Analysis of Attention Matrix

ArXi:2510.06685v2 Announce Type: replace-cross Self-attention layers have become fundamental building blocks of modern deep neural networks, yet their theoretical understanding remains limited, particularly from the perspective of random matrix theory. In this work, we provide a rigorous analysis of the singular value spectrum of the attention matrix and establish the first Gaussian equivalence result for attention.