AI RESEARCH
HiP-LoRA: Budgeted Spectral Plasticity for Robust Low-Rank Adaptation
arXiv CS.LG
•
ArXi:2604.17751v1 Announce Type: new Adapting foundation models under resource budgets relies heavily on Parameter-Efficient Fine-Tuning (PEFT), with LoRA being a standard modular solution. However, LoRA suffers from spectral interference. Low-rank updates often concentrate energy on the leading singular directions of pretrained weights, perturbing general capabilities and causing catastrophic forgetting and fragile multi-adapter merging. To resolve this, we propose HiP-LoRA, a spectrum-aware adaptation framework.