Transfer Learning with Neural AutoML
Catherine Wong, Neil Houlsby, Yifeng Lu, Andrea Gesmundo
Abstract
We reduce the computational cost of Neural AutoML with transfer learning. AutoML relieves human effort by automating the design of ML algorithms. Neural AutoML has become popular for the design of deep learning architectures, however, this method has a high computation cost. To address this we propose Transfer Neural AutoML that uses knowledge from prior tasks to speed up network design. We extend RL-based architecture search methods to support parallel training on multiple tasks and then transfer the search strategy to new tasks. On language and image classification data, Transfer Neural AutoML reduces convergence time over single-task training by over an order of magnitude on many tasks.
BibTeX
@inproceedings{NEURIPS2018_bdb3c278,
author = {Wong, Catherine and Houlsby, Neil and Lu, Yifeng and Gesmundo, Andrea},
booktitle = {Advances in Neural Information Processing Systems},
editor = {S. Bengio and H. Wallach and H. Larochelle and K. Grauman and N. Cesa-Bianchi and R. Garnett},
pages = {},
publisher = {Curran Associates, Inc.},
title = {Transfer Learning with Neural AutoML},
url = {https://proceedings.neurips.cc/paper_files/paper/2018/file/bdb3c278f45e6734c35733d24299d3f4-Paper.pdf},
volume = {31},
year = {2018}
}