[R] Causal self-attention as a probabilistic model over embeddings

We’ve been working on a probabilistic interpretation of causal self-attention where token embeddings are treated as latent variables. In that view, the attention map induces a change-of-variables term, which leads to a barrier / degeneracy boundary in embedding space. The resulting picture is: a stability-margin interpretation of causal attention “ tokens,” i.e. the positions closest to the degeneracy boundary a simple MAP-style