AI RESEARCH
Sparse Feature Coactivation Reveals Causal Semantic Modules in Large Language Models
arXiv CS.CL
•
ArXi:2506.18141v3 Announce Type: replace We identify semantically coherent, context-consistent network components in large language models (LLMs) using coactivation of sparse autoencoder (SAE) features collected from just a handful of prompts. Focusing on concept-relation prediction tasks, we show that ablating these components for concepts (e.g., countries and words) and relations (e.g., capital city and translation language) changes model outputs in predictable ways, while amplifying these components induces counterfactual responses.