AI RESEARCH
What DINO saw: ALiBi positional encoding reduces positional bias in Vision Transformers
arXiv CS.CV
•
ArXi:2603.16840v1 Announce Type: new Vision transformers (ViTs) - especially feature foundation models like DINOv2 - learn rich representations useful for many downstream tasks. However, architectural choices (such as positional encoding) can lead to these models displaying positional biases and artefacts independent of semantic content. This makes zero-shot adaption difficult in fields like material science, where images are often cross-sections of homogeneous microstructure (i.e. having no preferred direction