Transformation of Dense and Sparse Text Representations
Wenpeng Hu, Mengyu Wang, Bing Liu, Feng Ji, Jinwen Ma, Dongyan Zhao
Abstract
Sparsity is regarded as a desirable property of representations, especially in terms of explanation. However, its usage has been limited due to the gap with dense representations. Most research progresses in NLP in recent years are based on dense representations. Thus the desirable property of sparsity cannot be leveraged. Inspired by Fourier Transformation, in this paper, we propose a novel Semantic Transformation method to bridge the dense and sparse spaces, which can facilitate the NLP research to shift from dense spaces to sparse spaces or to jointly use both spaces. Experiments using classification tasks and natural language inference task show that the proposed Semantic Transformation is effective.
BibTeX
@inproceedings{hu-etal-2020-transformation,
title = "Transformation of Dense and Sparse Text Representations",
author = "Hu, Wenpeng and
Wang, Mengyu and
Liu, Bing and
Ji, Feng and
Ma, Jinwen and
Zhao, Dongyan",
editor = "Scott, Donia and
Bel, Nuria and
Zong, Chengqing",
booktitle = "Proceedings of the 28th International Conference on Computational Linguistics",
month = dec,
year = "2020",
address = "Barcelona, Spain (Online)",
publisher = "International Committee on Computational Linguistics",
url = "https://aclanthology.org/2020.coling-main.290/",
doi = "10.18653/v1/2020.coling-main.290",
pages = "3257--3267"
}