Language Models are Injective and Hence Invertible

ArXi:2510.15511v4 Announce Type: replace-cross Transformer components such as non-linear activations and normalization are inherently non-injective, suggesting that different inputs could map to the same output and prevent exact recovery of the input from a model's representations. In this paper, we challenge this view. First, we prove mathematically that transformer language models mapping discrete input sequences to their corresponding sequence of continuous representations are injective and. therefore. lossless, a property established at initialization and preserved during.