Graph Masked Autoencoder Enhanced Predictor for Neural Architecture Search

Kun Jing; Jungang Xu; Pengfei Li

IJCAI 2022poster19 citations

Graph Masked Autoencoder Enhanced Predictor for Neural Architecture Search

Abstract

Performance estimation of neural architecture is a crucial component of neural architecture search (NAS). Meanwhile, neural predictor is a current mainstream performance estimation method. However, it is a challenging task to train the predictor with few architecture evaluations for efficient NAS. In this paper, we propose a graph masked autoencoder (GMAE) enhanced predictor, which can reduce the dependence on supervision data by self-supervised pre-training with untrained architectures. We compare our GMAE-enhanced predictor with existing predictors in different search spaces, and experimental results show that our predictor has high query utilization. Moreover, GMAE-enhanced predictor with different search strategies can discover competitive architectures in different search spaces. Code and supplementary materials are available at https://github.com/kunjing96/GMAENAS.git.

Machine Learning: Automated Machine LearningComputer Vision: Recognition (object detection, categorization)Machine Learning: Learning Graphical ModelsMachine Learning: Self-supervised Learning

BibTeX

@inproceedings{ijcai2022p432,
  title     = {Graph Masked Autoencoder Enhanced Predictor for Neural Architecture Search},
  author    = {Jing, Kun and Xu, Jungang and Li, Pengfei},
  booktitle = {Proceedings of the Thirty-First International Joint Conference on
               Artificial Intelligence, {IJCAI-22}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {Lud De Raedt},
  pages     = {3114--3120},
  year      = {2022},
  month     = {7},
  note      = {Main Track},
  doi       = {10.24963/ijcai.2022/432},
  url       = {https://doi.org/10.24963/ijcai.2022/432},
}