Express Your Doubts -- Probabilistic World Modeling Should not be Based on Token logprobs

ArXi:2505.02072v2 Announce Type: replace-cross Language modeling has shifted in recent years from a distribution over strings to prediction models with textual inputs and outputs for general-purpose tasks. This position paper highlights the often overlooked implications of this shift for the use of large language models (LLMs) as probability estimators, especially for world probabilities. In light of the theoretical distinction between distribution estimation and response prediction, we examine.