2026
Reformulation for Pretraining Data Augmentation
ICLR 2026poster
Despite the impressive capabilities of large language models across various tasks, their continued scaling is severely hampered not only by data scarcity but also by the performance degradation associated with excessive data repetition during training. To overcome this critical bottleneck, we introd…