MorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep Networks
Ariel Gordon, Elad Eban, Ofir Nachum, Bo Chen, Hao Wu, Tien-Ju Yang, Edward Choi
Abstract
We present MorphNet, an approach to automate the design of neural network structures. MorphNet iteratively shrinks and expands a network, shrinking via a resource-weighted sparsifying regularizer on activations and expanding via a uniform multiplicative factor on all layers. In contrast to previous approaches, our method is scalable to large networks, adaptable to specific resource constraints (e.g. the number of floating-point operations per inference), and capable of increasing the network’s performance. When applied to standard network architectures on a wide variety of datasets, our approach discovers novel structures in each domain, obtaining higher performance while respecting the resource constraint.
BibTeX
@inproceedings{cvpr2018_morphnetfastsimp,
title = {MorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep Networks},
author = {Ariel Gordon and Elad Eban and Ofir Nachum and Bo Chen and Hao Wu and Tien-Ju Yang and Edward Choi},
booktitle = {CVPR 2018},
year = {2018}
}