Kishore Papineni

Promises and Pitfalls of Generative Masked Language Modeling: Theoretical Framework and Practical Guidelines

Autoregressive language models are the currently dominant paradigm for text generation, however they have some fundamental limitations that cannot be remedied by scale---for example inherently sequential and unidirectional generation. While alternate classes of models have been explored, we have lim…