AI RESEARCH
Self-Distillation of Hidden Layers for Self-Supervised Representation Learning
arXiv CS.LG
•
ArXi:2603.15553v1 Announce Type: cross The landscape of self-supervised learning (SSL) is currently dominated by generative approaches (e.g., MAE) that reconstruct raw low-level data, and predictive approaches (e.g., I-JEPA) that predict high-level abstract embeddings. While generative methods provide strong grounding, they are computationally inefficient for high-redundancy modalities like imagery, and their