AI RESEARCH
Self-Distillation is Optimal Among Spectral Shrinkage Estimators in Spiked Covariance Models
arXiv CS.LG
•
ArXi:2605.17778v1 Announce Type: cross Self-distillation has emerged as a promising technique for improving model performance in modern machine learning systems. We develop the statistical foundations of self-distillation in spiked covariance models, by