AI RESEARCH
R\'enyi Attention Entropy for Patch Pruning
arXiv CS.LG
•
ArXi:2604.03803v1 Announce Type: cross Transformers are strong baselines in both vision and language because self-attention captures long-range dependencies across tokens. However, the cost of self-attention grows quadratically with the number of tokens. Patch pruning mitigates this cost by estimating per-patch importance and removing redundant patches. To identify informative patches for pruning, we