Matched-Learning-Rate Analysis of Attention Drift and Transfer Retention in Fine-Tuned CLIP

ArXi:2604.16410v1 Announce Type: new CLIP adaptation can improve in-domain accuracy while degrading out-of-domain transfer, but comparisons between Full Fine-Tuning (Full FT) and LoRA are often confounded by different learning-rate conventions. We study how adaptation method and optimization scale jointly shape attention drift and transfer retention in CLIP using a controlled matched-learning-rate comparison of Full FT and LoRA.