2024
Consistent Bidirectional Language Modelling: Expressive Power and Representational Conciseness
EMNLP 2024main
The inability to utilise future contexts and the pre-determined left-to-right generation order are major limitations of unidirectional language models. Bidirectionality has been introduced to address those deficiencies. However, a crucial shortcoming of bidirectional language models is the potential…