2026
Prior-based Noisy Text Data Filtering: Fast and Strong Alternative For Perplexity
ICLR 2026poster
As large language models (LLMs) are pretrained on massive web corpora, careful selection of data becomes essential to ensure effective and efficient learning. While perplexity (PPL)-based filtering has demonstrated strong performance, it suffers from drawbacks: substantial time costs and inherent un…