Beyond Factor Aggregation: Gauge-Aware Low-Rank Server Representations for Federated LoRA

ArXi:2605.06733v1 Announce Type: cross Federated LoRA enables parameter-efficient adaptation of large language models under decentralized data and limited client resources. However, directly averaging LoRA factors is representation-dependent: the same intrinsic update admits infinitely many gauge-equivalent factorizations, so factor-level aggregation can change under arbitrary coordinate choices while the underlying update remains unchanged. This reveals a semantic mismatch in existing federated LoRA aggregation rules.