AI RESEARCH
Fragile Knowledge, Robust Instruction-Following: The Width Pruning Dichotomy in Llama-3.2
arXiv CS.AI
•
ArXi:2512.22671v2 Announce Type: replace-cross Structured width pruning of GLU-MLP layers, guided by the Maximum Absolute Weight (MAW) criterion, reveals a systematic dichotomy in how reducing the expansion ratio affects different model capabilities. While performance on tasks relying on parametric knowledge (e.g., MMLU, GSM8K) and perplexity metrics degrades predictably, instruction-following capabilities improve substantially (+46% to +75% in IFEval for Llama-3.2-1B and 3B models), and multi-step reasoning remains robust.