GREATS: Online Selection of High-Quality Data for LLM Training in Every Iteration
Jiachen T. Wang, Tong Wu, Dawn Song, Prateek Mittal, Ruoxi Jia
Abstract
Online batch selection methods offer an adaptive alternative to static training data selection by dynamically selecting data batches during training. However, existing methods either rely on impractical reference models or simple heuristics that may not capture true data informativeness. To address these limitations, we propose \emph{GREedy Approximation Taylor Selection} (GREATS), a principled and efficient online batch selection method that applies greedy algorithm to optimize the data batch quality approximated by Taylor expansion. We develop a series of techniques to scale GREATS to large-scale model training. Extensive experiments with large language models (LLMs) demonstrate that GREATS significantly improves training convergence speed and generalization performance.
BibTeX
@inproceedings{
wang2024greats,
title={{GREATS}: Online Selection of High-Quality Data for {LLM} Training in Every Iteration},
author={Jiachen T. Wang and Tong Wu and Dawn Song and Prateek Mittal and Ruoxi Jia},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=232VcN8tSx}
}