A Probabilistic Model for Dirty Multi-task Feature Selection
Daniel Hernandez-Lobato, Jose Miguel Hernandez-Lobato, Zoubin Ghahramani
Abstract
Multi-task feature selection methods often make the hypothesis that learning tasks share relevant and irrelevant features. However, this hypothesis may be too restrictive in practice. For example, there may be a few tasks with specific relevant and irrelevant features (outlier tasks). Similarly, a few of the features may be relevant for only some of the tasks (outlier features). To account for this, we propose a model for multi-task feature selection based on a robust prior distribution that introduces a set of binary latent variables to identify outlier tasks and outlier features. Expectation propagation can be used for efficient approximate inference under the proposed prior. Several experiments show that a model based on the new robust prior provides better predictive performance than other benchmark methods.
BibTeX
@InProceedings{pmlr-v37-hernandez-lobatoa15,
title = {A Probabilistic Model for Dirty Multi-task Feature Selection},
author = {Hernandez-Lobato, Daniel and Hernandez-Lobato, Jose Miguel and Ghahramani, Zoubin},
booktitle = {Proceedings of the 32nd International Conference on Machine Learning},
pages = {1073--1082},
year = {2015},
editor = {Bach, Francis and Blei, David},
volume = {37},
series = {Proceedings of Machine Learning Research},
address = {Lille, France},
month = {07--09 Jul},
publisher = {PMLR},
pdf = {http://proceedings.mlr.press/v37/hernandez-lobatoa15.pdf},
url = {https://proceedings.mlr.press/v37/hernandez-lobatoa15.html},
abstract = {Multi-task feature selection methods often make the hypothesis that learning tasks share relevant and irrelevant features. However, this hypothesis may be too restrictive in practice. For example, there may be a few tasks with specific relevant and irrelevant features (outlier tasks). Similarly, a few of the features may be relevant for only some of the tasks (outlier features). To account for this, we propose a model for multi-task feature selection based on a robust prior distribution that introduces a set of binary latent variables to identify outlier tasks and outlier features. Expectation propagation can be used for efficient approximate inference under the proposed prior. Several experiments show that a model based on the new robust prior provides better predictive performance than other benchmark methods.}
}