Transformer Interpretability from Perspective of Attention and Gradient

ArXi:2605.11392v1 Announce Type: new Although researchers' attention is focused on the performance of Transformer models, the interpretation of Transformer can never be ignored. Gradient is widely utilized in Transformer interpretation. From the perspective of attention and gradient, we conduct an in-depth study of Transformer interpretation and propose a method to achieve it by guiding the gradient direction, or precisely, the attention direction.