AI RESEARCH
SPoT: Subpixel Placement of Tokens in Vision Transformers
arXiv CS.LG
•
ArXi:2507.01654v2 Announce Type: replace-cross Vision Transformers naturally accommodate sparsity, yet standard tokenization methods confine features to discrete patch grids. This constraint prevents models from fully exploiting sparse regimes, forcing awkward compromises. We propose Subpixel Placement of Tokens (SPoT), a novel tokenization strategy that positions tokens continuously within images, effectively sidestepping grid-based limitations.