Sarah Marzen

Transformers Represent Belief State Geometry in their Residual Stream

What computational structure are we building into large language models when we train them on next-token prediction? Here, we present evidence that this structure is given by the meta-dynamics of belief updating over hidden states of the data- generating process. Leveraging the theory of optimal pre…