AI RESEARCH
When Does Sparse MoE Help in Vision? The Role of Backbone Compute Leverage in Sparse Routing
arXiv CS.LG
•
ArXi:2605.15484v1 Announce Type: cross Mixture-of-Experts (MoE) networks promise favorable accuracy-compute trade-offs, yet practical vision deployments are hindered by expert collapse and limited end-to-end efficiency gains. We study when sparse top-$k$ routing with hard capacity constraints helps in vision classification, evaluated under multi-seed protocols on four benchmarks (CIFAR-10/100, Tiny-ImageNet, ImageNet-1K