A Geometric Perspective on Next-Token Prediction in Large Language Models: Three Emerging Phases

ArXi:2605.09011v1 Announce Type: cross We investigate the geometry of predictive information across the layers of large language models (LLMs). We repurpose representation lenses-learned affine maps trained to predict the next token from intermediate residual streams-as geometric diagnostic tools. Rather than asking what the model predicts at each layer, we ask where predictive information resides and how it evolves across depth.