AI RESEARCH
Taming Outlier Tokens in Diffusion Transformers
arXiv CS.AI
•
ArXi:2605.05206v1 Announce Type: cross We study outlier tokens in Diffusion Transformers (DiTs) for image generation. Prior work has shown that Vision Transformers (ViTs) can produce a small number of high-norm tokens that attract disproportionate attention while carrying limited local information, but their role in generative models remains underexplored.