Inductive Two-Layer Modeling with Parametric Bregman Transfer
Vignesh Ganapathiraman, Zhan Shi, Xinhua Zhang, Yaoliang Yu
Abstract
Latent prediction models, exemplified by multi-layer networks, employ hidden variables that automate abstract feature discovery. They typically pose nonconvex optimization problems and effective semi-definite programming (SDP) relaxations have been developed to enable global solutions (Aslan et al., 2014).However, these models rely on nonparametric training of layer-wise kernel representations, and are therefore restricted to transductive learning which slows down test prediction. In this paper, we develop a new inductive learning framework for parametric transfer functions using matching losses. The result for ReLU utilizes completely positive matrices, and the inductive learner not only delivers superior accuracy but also offers an order of magnitude speedup over SDP with constant approximation guarantees.
BibTeX
@InProceedings{pmlr-v80-ganapathiraman18a,
title = {Inductive Two-Layer Modeling with Parametric {B}regman Transfer},
author = {Ganapathiraman, Vignesh and Shi, Zhan and Zhang, Xinhua and Yu, Yaoliang},
booktitle = {Proceedings of the 35th International Conference on Machine Learning},
pages = {1636--1645},
year = {2018},
editor = {Dy, Jennifer and Krause, Andreas},
volume = {80},
series = {Proceedings of Machine Learning Research},
month = {10--15 Jul},
publisher = {PMLR},
pdf = {http://proceedings.mlr.press/v80/ganapathiraman18a/ganapathiraman18a.pdf},
url = {https://proceedings.mlr.press/v80/ganapathiraman18a.html},
abstract = {Latent prediction models, exemplified by multi-layer networks, employ hidden variables that automate abstract feature discovery. They typically pose nonconvex optimization problems and effective semi-definite programming (SDP) relaxations have been developed to enable global solutions (Aslan et al., 2014).However, these models rely on nonparametric training of layer-wise kernel representations, and are therefore restricted to transductive learning which slows down test prediction. In this paper, we develop a new inductive learning framework for parametric transfer functions using matching losses. The result for ReLU utilizes completely positive matrices, and the inductive learner not only delivers superior accuracy but also offers an order of magnitude speedup over SDP with constant approximation guarantees.}
}