AI RESEARCH
LoRA-DA: Data-Aware Initialization for Low-Rank Adaptation via Asymptotic Analysis
arXiv CS.LG
•
ArXi:2510.24561v2 Announce Type: replace LoRA has become a widely adopted method for PEFT, and its initialization methods have attracted increasing attention. However, existing methods have notable limitations: many methods do not incorporate target-domain data, while gradient-based methods exploit data only at a shallow level by relying on one-step gradient decomposition. In this paper, we establish a theoretical framework for data-aware LoRA initialization.