Continual Fine-Tuning with Provably Accurate and Parameter-Free Task Retrieval

ArXi:2603.13235v1 Announce Type: new Continual fine-tuning aims to adapt a pre-trained backbone to new tasks sequentially while preserving performance on earlier tasks whose data are no longer available. Existing approaches fall into two categories which include input- and parameter-adaptation. Input-adaptation methods rely on retrieving the most relevant prompts at test time, but require continuously learning a retrieval function that is prone to forgetting.