FLoRG: Federated Fine-tuning with Low-rank Gram Matrices and Procrustes Alignment

ArXi:2602.17095v2 Announce Type: replace-cross Parameter-efficient fine-tuning techniques such as low-rank adaptation (LoRA) enable large language models (LLMs) to adapt to downstream tasks efficiently. Federated learning (FL) further facilitates this process by enabling collaborative fine-tuning across distributed clients without sharing private data. However, the use of two separate low-rank matrices in LoRA for federated fine-tuning