Why Exposure Bias Matters: An Imitation Learning Perspective of Error Accumulation in Language Generation
Current language generation models suffer from issues such as repetition, incoherence, and hallucinations. An often-repeated hypothesis for this brittleness of generation models is that it is caused by the training and the generation procedure mismatch, also referred to as exposure bias. In this pap…