AI RESEARCH
Same Geometry, Opposite Noise: Transformer Magnitude Representations Lack Scalar Variability
arXiv CS.CL
•
ArXi:2604.04469v1 Announce Type: new Scalar variability -- the finding that representational noise scales proportionally with magnitude, producing a constant coefficient of variation -- is a hallmark of biological magnitude systems. We tested whether transformer language models exhibit this property by analysing the dispersion of hidden-state representations across carrier sentences for 26 numerical magnitudes in three 7-8B parameter models (Llama-3-8B-Instruct, Mistral-7B-Instruct-v0.3, Llama-3-8B-Base; data from Cacioli, 2026.