AI RESEARCH
Scaling Multi-Node Mixture-of-Experts Inference Using Expert Activation Patterns
arXiv CS.LG
•
ArXi:2604.23150v1 Announce Type: new Most recent state-of-the-art (SOTA) large language models (LLMs) use Mixture-of-Experts (MoE) architectures to scale model capacity without proportional per-token compute, enabling higher-quality outputs at manageable serving costs. However, MoE inference at scale is fundamentally bottlenecked by expert load imbalance and inefficient token routing, especially in multi-node deployments where tokens are not guaranteed to be routed to local experts, resulting in significant inter-node all-to-all communication overhead.