Andrea W Wen-Yi

Hyperpolyglot LLMs: Cross-Lingual Interpretability in Token Embeddings

Cross-lingual transfer learning is an important property of multilingual large language models (LLMs). But how do LLMs represent relationships between languages? Every language model has an input layer that maps tokens to vectors. This ubiquitous layer of language models is often overlooked. We fin…