AI RESEARCH
TinySSL: Distilled Self-Supervised Pretraining for Sub-Megabyte MCU Models
arXiv CS.AI
•
ArXi:2605.08241v1 Announce Type: cross Self-supervised learning (SSL) has transformed representation learning for large models, yet remains unexplored for microcontroller (MCU)-class models with fewer than 500K parameters. We identify three obstacles at this scale -- projection head dominance, representation bottleneck, and augmentation sensitivity -- and propose Capacity-Aware Distilled Self-Supervised Learning (CA-DSSL), a teacher-guided framework that overcomes them without labels or text supervision.