AI RESEARCH
FaaSMoE: A Serverless Framework for Multi-Tenant Mixture-of-Experts Serving
arXiv CS.LG
•
ArXi:2604.26881v1 Announce Type: cross Mixture-of-Experts (MoE) models offer high capacity with efficient inference cost by activating a small subset of expert models per input. However, deploying MoE models requires all experts to reside in memory, creating a gap between the resource used by activated experts and the provisioned resources. This underutilization is further pronounced in multi-tenant scenarios. In this paper, we propose FaaSMoE, a multi-tenant MoE serving architecture built on Function-as-a-Service (FaaS) platforms.