Perturbation: A simple and efficient adversarial tracer for representation learning in language models

ArXi:2603.23821v1 Announce Type: cross Linguistic representation learning in deep neural language models (LMs) has been studied for decades, for both practical and theoretical reasons. However, finding representations in LMs remains an unsolved problem, in part due to a dilemma between enforcing implausible constraints on representations (e.g., linearity; Arora 2024) and trivializing the notion of representation altogether (Sutter, 2025). Here we escape this dilemma by reconceptualizing representations not as patterns of activation but as conduits for learning.