Task Ecologies and the Evolution of World-Tracking Representations in Large Language Models

ArXi:2604.05469v1 Announce Type: cross We study language models as evolving model organisms and ask when autoregressive next-token learning selects for world-tracking representations. For any encoding of latent world states, the Bayes-optimal next-token cross-entropy decomposes into the irreducible conditional entropy plus a Jensen--Shannon excess term. That excess vanishes if and only if the encoding preserves the