AI RESEARCH
Replay-Free Continual Low-Rank Adaptation with Dynamic Memory
arXiv CS.LG
•
ArXi:2411.00623v4 Announce Type: replace-cross We revisit continual learning~(CL), which enables pre-trained vision transformers (ViTs) to sequentially fine-tune on new downstream tasks over time. However, as the scale of these models increases, catastrophic forgetting remains a serious challenge. Recent studies highlight a crossover between CL techniques and parameter-efficient fine-tuning (PEFT), which focuses on fine-tuning only a small set of trainable parameters to adapt to downstream tasks, such as low-rank adaptation (LoRA