A Closer Look at the Training Strategy for Modern Meta-Learning

JIAXIN CHEN; Xiao-Ming Wu; Yanke Li; Qimai LI; Li-Ming Zhan; Fu-lai Chung

NeurIPS 2020poster43 citations

A Closer Look at the Training Strategy for Modern Meta-Learning

JIAXIN CHEN, Xiao-Ming Wu, Yanke Li, Qimai LI, Li-Ming Zhan, Fu-lai Chung

Abstract

The support/query (S/Q) episodic training strategy has been widely used in modern meta-learning algorithms and is believed to improve their generalization ability to test environments. This paper conducts a theoretical investigation of this training strategy on generalization. From a stability perspective, we analyze the generalization error bound of generic meta-learning algorithms trained with such strategy. We show that the S/Q episodic training strategy naturally leads to a counterintuitive generalization bound of $O(1/\sqrt{n})$, which only depends on the task number $n$ but independent of the inner-task sample size $m$. Under the common assumption $m<

BibTeX

@inproceedings{NEURIPS2020_0415740e,
 author = {CHEN, JIAXIN and Wu, Xiao-Ming and Li, Yanke and LI, Qimai and Zhan, Li-Ming and Chung, Fu-lai},
 booktitle = {Advances in Neural Information Processing Systems},
 editor = {H. Larochelle and M. Ranzato and R. Hadsell and M.F. Balcan and H. Lin},
 pages = {396--406},
 publisher = {Curran Associates, Inc.},
 title = {A Closer Look at the Training Strategy for Modern Meta-Learning},
 url = {https://proceedings.neurips.cc/paper_files/paper/2020/file/0415740eaa4d9decbc8da001d3fd805f-Paper.pdf},
 volume = {33},
 year = {2020}
}