NeurIPS 2017spotlight7 citations
On Separability of Loss Functions, and Revisiting Discriminative Vs Generative Models
Adarsh Prasad, Alexandru Niculescu-Mizil, Pradeep K Ravikumar
Abstract
We revisit the classical analysis of generative vs discriminative models for general exponential families, and high-dimensional settings. Towards this, we develop novel technical machinery, including a notion of separability of general loss functions, which allow us to provide a general framework to obtain l∞ convergence rates for general M-estimators. We use this machinery to analyze l∞ and l2 convergence rates of generative and discriminative models, and provide insights into their nuanced behaviors in high-dimensions. Our results are also applicable to differential parameter estimation, where the quantity of interest is the difference between generative model parameters.
BibTeX
@inproceedings{NIPS2017_50cf0763,
author = {Prasad, Adarsh and Niculescu-Mizil, Alexandru and Ravikumar, Pradeep K},
booktitle = {Advances in Neural Information Processing Systems},
editor = {I. Guyon and U. Von Luxburg and S. Bengio and H. Wallach and R. Fergus and S. Vishwanathan and R. Garnett},
pages = {},
publisher = {Curran Associates, Inc.},
title = {On Separability of Loss Functions, and Revisiting Discriminative Vs Generative Models},
url = {https://proceedings.neurips.cc/paper_files/paper/2017/file/50cf0763d8eb871776d4f28b39deb564-Paper.pdf},
volume = {30},
year = {2017}
}