Prabhakar Raghavan

Latent Concept Disentanglement in Transformer-based Language Models

When large language models (LLMs) use in-context learning (ICL) to solve a new task, they must infer latent concepts from demonstration examples. This raises the question of whether and how transformers represent latent structures as part of their computation. Our work experiments with several contr…