AI RESEARCH
Influence Malleability in Linearized Attention: Dual Implications of Non-Convergent NTK Dynamics
arXiv CS.LG
•
ArXi:2603.13085v1 Announce Type: new Understanding the theoretical foundations of attention mechanisms remains challenging due to their complex, non-linear dynamics. This work reveals a fundamental trade-off in the learning dynamics of linearized attention. Using a linearized attention mechanism with exact correspondence to a data-dependent Gram-induced kernel, both empirical and theoretical analysis through the Neural Tangent Kernel (NTK) framework shows that linearized attention does not converge to its infinite-width NTK limit, even at large widths.