AI RESEARCH
SoftSAE: Dynamic Top-K Selection for Adaptive Sparse Autoencoders
arXiv CS.LG
•
ArXi:2605.06610v1 Announce Type: new Sparse Autoencoders (SAEs) have become an important tool in mechanistic interpretability, helping to analyze internal representations in both Large Language Models (LLMs) and Vision Transformers (ViTs). By decomposing polysemantic activations into sparse sets of monosemantic features, SAEs aim to translate neural network computations into human-understandable concepts. However, common architectures such as TopK SAEs rely on a fixed sparsity level.