Improving Word Embeddings through Iterative Refinement of Word- and Character-level Models
Phong Ha, Shanshan Zhang, Nemanja Djuric, Slobodan Vucetic
Abstract
Embedding of rare and out-of-vocabulary (OOV) words is an important open NLP problem. A popular solution is to train a character-level neural network to reproduce the embeddings from a standard word embedding model. The trained network is then used to assign vectors to any input string, including OOV and rare words. We enhance this approach and introduce an algorithm that iteratively refines and improves both word- and character-level models. We demonstrate that our method outperforms the existing algorithms on 5 word similarity data sets, and that it can be successfully applied to job title normalization, an important problem in the e-recruitment domain that suffers from the OOV problem.
BibTeX
@inproceedings{ha-etal-2020-improving,
title = "Improving Word Embeddings through Iterative Refinement of Word- and Character-level Models",
author = "Ha, Phong and
Zhang, Shanshan and
Djuric, Nemanja and
Vucetic, Slobodan",
editor = "Scott, Donia and
Bel, Nuria and
Zong, Chengqing",
booktitle = "Proceedings of the 28th International Conference on Computational Linguistics",
month = dec,
year = "2020",
address = "Barcelona, Spain (Online)",
publisher = "International Committee on Computational Linguistics",
url = "https://aclanthology.org/2020.coling-main.104/",
doi = "10.18653/v1/2020.coling-main.104",
pages = "1204--1213"
}