Learning Rate Matters: Vanilla LoRA May Suffice for LLM Fine-tuning

ArXi:2602.04998v2 Announce Type: replace-cross Low-Rank Adaptation (LoRA) is the prevailing approach for efficient large language model (LLM) fine-tuning. Building on this paradigm, recent studies have proposed alternative initialization strategies, architectural modifications, and optimization adjustments, reporting substantial improvements over vanilla LoRA. However, these gains are often nstrated under fixed or narrowly tuned hyperparameter settings, despite the known sensitivity of neural networks to