LoFT: Low-Rank Adaptation That Behaves Like Full Fine-Tuning

ArXi:2505.21289v2 Announce Type: replace Large pre-trained models are commonly adapted to downstream tasks using parameter-efficient fine-tuning methods such as Low-Rank Adaptation (LoRA), which injects small trainable low-rank matrices instead of updating all weights. While LoRA dramatically reduces trainable parameters with little overhead, it can still underperform full fine-tuning in accuracy and often converges slowly. We