Context Selection for Embedding Models
Liping Liu, Francisco Ruiz, Susan Athey, David Blei
Abstract
Word embeddings are an effective tool to analyze language. They have been recently extended to model other types of data beyond text, such as items in recommendation systems. Embedding models consider the probability of a target observation (a word or an item) conditioned on the elements in the context (other words or items). In this paper, we show that conditioning on all the elements in the context is not optimal. Instead, we model the probability of the target conditioned on a learned subset of the elements in the context. We use amortized variational inference to automatically choose this subset. Compared to standard embedding models, this method improves predictions and the quality of the embeddings.
BibTeX
@inproceedings{NIPS2017_7884a965,
author = {Liu, Liping and Ruiz, Francisco and Athey, Susan and Blei, David},
booktitle = {Advances in Neural Information Processing Systems},
editor = {I. Guyon and U. Von Luxburg and S. Bengio and H. Wallach and R. Fergus and S. Vishwanathan and R. Garnett},
pages = {},
publisher = {Curran Associates, Inc.},
title = {Context Selection for Embedding Models},
url = {https://proceedings.neurips.cc/paper_files/paper/2017/file/7884a9652e94555c70f96b6be63be216-Paper.pdf},
volume = {30},
year = {2017}
}