Alex Mallen

When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories

Despite their impressive performance on diverse tasks, large language models (LMs) still struggle with tasks requiring rich world knowledge, implying the difficulty of encoding a wealth of world knowledge in their parameters. This paper aims to understand LMs’ strengths and limitations in memorizing…