Liu Peng

Robustness of Probabilistic Models to Low-Quality Data: A Multi-Perspective Analysis

A systematic, comparative investigation into the effects of low-quality data reveals a stark spectrum of robustness across modern probabilistic models. We find that autoregressive language models, from token prediction to sequence-to-sequence tasks, are remarkably resilient (for GPT-2, test NLL i…