2026
GneissWeb: Preparing High Quality Data for LLMs at Scale
Hajar Emami Gohari, Swanand Ravindra Kadhe, Yousaf Shah, Constantin M Adam, Abdulhamid Adebayo, Praneet Adusumilli +26
ICLR 2026poster
Data quantity and quality play a vital role in determining the performance of Large Language Models (LLMs). High-quality data, in particular, can significantly boost the LLM's ability to generalize on a wide range of downstream tasks. In this paper, we introduce **GneissWeb**, a large dataset of aro…