Collaborative Parameter Learning: Mitigating Forgetting via Parameter-Level Gradient Analysis

ArXi:2601.21577v2 Announce Type: replace Catastrophic forgetting during knowledge injection impairs the ability of large language models to acquire new knowledge without overwriting previously mastered knowledge. Recent studies analyze forgetting from a gradient similarity perspective and mitigate forgetting through vector projection. However, these methods primarily characterize gradient similarity at the aggregate direction level, leaving the parameter wise contributions to forgetting underexplored.