Words or Characters? Fine-grained Gating for Reading Comprehension
Zhilin Yang, Bhuwan Dhingra, Ye Yuan, Junjie Hu, William W. Cohen, Ruslan Salakhutdinov
Abstract
Previous work combines word-level and character-level representations using concatenation or scalar weighting, which is suboptimal for high-level tasks like reading comprehension. We present a fine-grained gating mechanism to dynamically combine word-level and character-level representations based on properties of the words. We also extend the idea of fine-grained gating to modeling the interaction between questions and paragraphs for reading comprehension. Experiments show that our approach can improve the performance on reading comprehension tasks, achieving new state-of-the-art results on the Children's Book Test and Who Did What datasets. To demonstrate the generality of our gating mechanism, we also show improved results on a social media tag prediction task.
BibTeX
@inproceedings{
yang2017words,
title={Words or Characters? Fine-grained Gating for Reading Comprehension},
author={Zhilin Yang and Bhuwan Dhingra and Ye Yuan and Junjie Hu and William W. Cohen and Ruslan Salakhutdinov},
booktitle={International Conference on Learning Representations},
year={2017},
url={https://openreview.net/forum?id=B1hdzd5lg}
}