Maximilian Kaufmann

The Reversal Curse: LLMs trained on “A is B” fail to learn “B is A”

We expose a surprising failure of generalization in auto-regressive large language models (LLMs). If a model is trained on a sentence of the form ''_A_ is _B_'', it will not automatically generalize to the reverse direction ''_B_ is _A_''. This is the **Reversal Curse**. For instance, if a model is…