AI RESEARCH
Sparse Memory Finetuning as a Low-Forgetting Alternative to LoRA and Full Finetuning
arXiv CS.LG
•
ArXi:2605.03229v1 Announce Type: cross Adapting a pretrained language model to a new task often hurts the general capabilities it already had, a problem known as catastrophic forgetting. Sparse Memory Finetuning (SMF) tries to avoid this by adding key-value memory layers to the model and, on each