Ying Jiangyong

P3LM: Probabilistically Permuted Prophet Language Modeling for Generative Pre-Training

Conventional autoregressive left-to-right (L2R) sequence generation faces two issues during decoding: limited to unidirectional target sequence modeling, and constrained on strong local dependencies.To address the aforementioned problem, we propose P3LM, a probabilistically permuted prophet language…