AI RESEARCH
MoE-Sieve: Routing-Guided LoRA for Efficient MoE Fine-Tuning
arXiv CS.LG
•
ArXi:2603.24044v1 Announce Type: new Standard LoRA fine-tuning of Mixture-of-Experts (MoE) models applies adapters to every expert, yet our profiling shows that per-layer expert routing is highly skewed: a small subset of experts handles most tokens in each layer, while many others are rarely activated ("cold"). We propose MoE-Sieve, a simple routing-guided framework for LoRA fine-tuning, and pair it with a systematic profiling study of expert routing across architectures and tasks.