AI RESEARCH
R\'enyi Entropy: A New Token Pruning Metric for Vision Transformers
arXiv CS.CV
•
ArXi:2603.27900v1 Announce Type: new Vision Transformers (ViTs) achieve state-of-the-art performance but suffer from the $O(N^2)$ complexity of self-attention, making inference costly for high-resolution inputs. To address this bottleneck, token pruning has emerged as a critical technique to accelerate inference. Most existing methods rely on the [CLS] token to estimate patch importance. However, we argue that the [CLS] token can be unreliable in early layers where semantic representations are still immature.