NeurIPS 2019poster8 citations
Faster Boosting with Smaller Memory
Julaiti Alafate, Yoav S Freund
Abstract
State-of-the-art implementations of boosting, such as XGBoost and LightGBM, can process large training sets extremely fast. However, this performance requires that the memory size is sufficient to hold a 2-3 multiple of the training set size. This paper presents an alternative approach to implementing the boosted trees, which achieves a significant speedup over XGBoost and LightGBM, especially when the memory size is small. This is achieved using a combination of three techniques: early stopping, effective sample size, and stratified sampling. Our experiments demonstrate a 10-100 speedup over XGBoost when the training data is too large to fit in memory.
BibTeX
@inproceedings{NEURIPS2019_3ffebb08,
author = {Alafate, Julaiti and Freund, Yoav S},
booktitle = {Advances in Neural Information Processing Systems},
editor = {H. Wallach and H. Larochelle and A. Beygelzimer and F. d\textquotesingle Alch\'{e}-Buc and E. Fox and R. Garnett},
pages = {},
publisher = {Curran Associates, Inc.},
title = {Faster Boosting with Smaller Memory},
url = {https://proceedings.neurips.cc/paper_files/paper/2019/file/3ffebb08d23c609875d7177ee769a3e9-Paper.pdf},
volume = {32},
year = {2019}
}