Deep Fried Convnets
Zichao Yang, Marcin Moczulski, Misha Denil, Nando de Freitas, Alex Smola, Le Song, Ziyu Wang
Abstract
The fully connected layers of a deep convolutional neural network typically contain over 90% of the network parameters, and consume the majority of the memory required to store the network. Reducing the number of parameters while preserving predictive performance is critically important for deploying deep neural networks in memory constrained environments such as GPUs or embedded devices. In this paper we show how kernel methods, in particular a single Fastfood layer, can be used to replace the fully connected layers in a deep convolutional neural network. This deep fried network is end-to-end trainable in conjunction with convolutional layers. Our new architecture substantially reduces the memory footprint of convolutional networks trained on MNIST and ImageNet with no drop in predictive performance
BibTeX
@inproceedings{iccv2015_deepfriedconvnet,
title = {Deep Fried Convnets},
author = {Zichao Yang and Marcin Moczulski and Misha Denil and Nando de Freitas and Alex Smola and Le Song and Ziyu Wang},
booktitle = {ICCV 2015},
year = {2015}
}