AI RESEARCH
LiME: Lightweight Mixture of Experts for Efficient Multimodal Multi-task Learning
arXiv CS.LG
•
ArXi:2604.02338v1 Announce Type: new MoE-PEFT methods combine Mixture of Experts with parameter-efficient fine-tuning for multi-task adaptation, but require separate adapters per expert causing trainable parameters to scale linearly with expert count and limiting applicability to adapter-based architectures. We propose LiME (Lightweight Mixture of Experts), which achieves expert specialization through lightweight modulation rather than adapter replication.