AI RESEARCH
Resting Neurons, Active Insights: Robustify Activation Sparsity for Large Language Models
arXiv CS.LG
•
ArXi:2512.12744v2 Announce Type: replace Activation sparsity offers a compelling route to accelerate large language model (LLM) inference by selectively suppressing hidden activations, yet existing approaches exhibit severe accuracy degradation at high sparsity. We show that this failure stems from representational instability: *activation sparsity disrupts input-dependent activation learned during pre