AI RESEARCH
DECO: Sparse Mixture-of-Experts with Dense-Comparable Performance on End-Side Devices
arXiv CS.LG
•
ArXi:2605.10933v1 Announce Type: new While Mixture-of-Experts (MoE) scales model capacity without proportionally increasing computation, its massive total parameter footprint creates significant storage and memory-access bottlenecks, which hinder efficient end-side deployment that simultaneously requires high performance, low computational cost, and small storage overhead. To achieve these properties, we present DECO, a sparse MoE architecture designed to match the performance of dense Transformers under identical total parameter budgets and.