AI RESEARCH
ReMix: Reinforcement routing for mixtures of LoRAs in LLM finetuning
arXiv CS.LG
•
ArXi:2603.10160v1 Announce Type: new Low-rank adapters (LoRAs) are a parameter-efficient finetuning technique that injects trainable low-rank matrices into pretrained models to adapt them to new tasks. Mixture-of-LoRAs models expand neural networks efficiently by routing each layer input to a small subset of specialized LoRAs of the layer. Existing Mixture-of-LoRAs routers assign a learned routing weight to each LoRA to enable end-to-end