Distilling Vision Transformers for Distortion-Robust Representation Learning

ArXi:2604.22529v1 Announce Type: new Self-supervised learning has achieved remarkable success in learning visual representations from clean data, yet remains challenging when clean observations are sparse or not available at all. In this paper, we nstrate that pretrained vision models can be leveraged to learn distortion-robust representations, which can then be effectively applied to downstream tasks operating on distorted observations.