AI RESEARCH
Tucker Attention: A generalization of approximate attention mechanisms
arXiv CS.AI
•
ArXi:2603.30033v1 Announce Type: cross The pursuit of reducing the memory footprint of the self-attention mechanism in multi-headed self attention (MHA) spawned a rich portfolio of methods, e.g., group-query attention (GQA) and multi-head latent attention (MLA). The methods leverage specialized low-rank factorizations across embedding dimensions or attention heads.