Hamidreza Saghir

Llama See, Llama Do: A Mechanistic Perspective on Contextual Entrainment and Distraction in LLMs

We observe a novel phenomenon, *contextual entrainment*, across a wide range of language models (LMs) and prompt settings, providing a new mechanistic perspective on how LMs become distracted by “irrelevant” contextual information in the input prompt. Specifically, LMs assign significantly higher lo…