AI RESEARCH
Design and Behavior of Sparse Mixture-of-Experts Layers in CNN-based Semantic Segmentation
arXiv CS.LG
•
ArXi:2604.13761v1 Announce Type: cross Sparse mixture-of-experts (MoE) layers have been shown to substantially increase model capacity without a proportional increase in computational cost and are widely used in transformer architectures, where they typically replace feed-forward network blocks. In contrast, integrating sparse MoE layers into convolutional neural networks (CNNs) remains inconsistent, with most prior work focusing on fine-grained MoEs operating at the filter or channel levels.