AI RESEARCH
Variance Is Not Importance: Structural Analysis of Transformer Compressibility Across Model Scales
arXiv CS.LG
•
ArXi:2604.20682v1 Announce Type: new We present a systematic empirical study of transformer compression through over 40 experiments on GPT-2 (124M parameters) and Mistral 7B (7.24B parameters). Our analysis covers spectral compression, block-level function replacement, rotation-based quantization, activation geometry, and adaptive early exit. We identify five structural properties relevant to compression.