AI RESEARCH
PACE: Marrying generalization in PArameter-efficient fine-tuning with Consistency rEgularization
arXiv CS.LG
•
ArXi:2409.17137v5 Announce Type: replace Parameter-Efficient Fine-Tuning (PEFT) effectively adapts pre-trained transformers to downstream tasks. However, the optimization of tasks performance often comes at the cost of generalizability in fine-tuned models. To address this issue, we theoretically connect smaller weight gradient norms during