AI RESEARCH
The Routing and Filtering Structure of Attention
arXiv CS.AI
•
ArXi:2605.18826v1 Announce Type: cross The attention interaction matrix $QK^{\top}$ contains two entangled computations: a skew-symmetric component that redistributes information between positions (routing) and a symmetric component that scales mutual relevance (filtering). We decompose 1776 heads across five pretrained transformers and find routing operating at low rank, well below the routing capacity allocated by the weight kernel. We