AI RESEARCH
Sensitivity-Positional Co-Localization in GQA Transformers
arXiv CS.CL
•
ArXi:2604.07766v1 Announce Type: new We investigate a fundamental structural question in Grouped Query Attention (GQA) transformers: do the layers most sensitive to task correctness coincide with the layers where positional encoding adaptation has the greatest leverage? We term this the co-localization hypothesis and test it on Llama 3.1 8B, a 32-layer GQA model with a 4:1 query-to-key-value head ratio. We