FedSDR: Federated Self-Distillation with Rectification

ArXi:2605.18028v1 Announce Type: new Federated fine-tuning of Large Language Models faces severe statistical heterogeneity. However, existing model-level defenses often overlook the root cause: intrinsic data distribution mismatches. In this work, we first establish Federated Self-Distillation (FedSD) as a fundamental and potent strategy. By projecting client representations into a smoothed ``model-understanding space,'' FedSD alone serves as a universal booster, nstrating superior performance over conventional algorithms.