AI RESEARCH
A Mechanistic Investigation of Supervised Fine Tuning
arXiv CS.AI
•
ArXi:2605.11426v1 Announce Type: new The cosine similarity between a large language model's hidden activations before and after Supervised Fine-Tuning (SFT) remains very high. This, at first glance, suggests that SFT leaves the model's activation geometry largely undisturbed. However, projecting both sets of activations through a Sparse Autoencoder (SAE) pretrained on the base model reveals that the underlying sparse latents diverge significantly. We