Gabor Melis — accepted AI-conference papers

2019

Variational Smoothing in Recurrent Neural Network Language Models

Lingpeng Kong, Gabor Melis, Wang Ling, Lei Yu, Dani Yogatama

ICLR 2019poster

We present a new theoretical perspective of data noising in recurrent neural network language models (Xie et al., 2017). We show that each variant of data noising is an instance of Bayesian recurrent neural networks with a particular variational distribution (i.e., a mixture of Gaussians whose weig…

Cited by 3 Source ↗PDF ↗Scholar ↗

2018

Memory Architectures in Recurrent Neural Network Language Models

Dani Yogatama, Yishu Miao, Gabor Melis, Wang Ling, Adhiguna Kuncoro, Chris Dyer +1

ICLR 2018poster

We compare and analyze sequential, random access, and stack memory architectures for recurrent neural network language models. Our experiments on the Penn Treebank and Wikitext-2 datasets show that stack-based memory architectures consistently achieve the best performance in terms of held out perple…

Cited by 66 Source ↗PDF ↗Scholar ↗