Position: Express Your Doubts — Probabilistic World Modeling Should not be Based on Token *logprobs*
Language modeling has shifted in recent years from a distribution over strings to prediction models with textual inputs and outputs for general-purpose tasks. This position paper highlights the often overlooked implications of this shift for the use of large language models (LLMs) as probability est…