COLING 2020main6 citations
Joint Persian Word Segmentation Correction and Zero-Width Non-Joiner Recognition Using BERT
Ehsan Doostmohammadi, Minoo Nassajian, Adel Rahimi
Abstract
Words are properly segmented in the Persian writing system; in practice, however, these writing rules are often neglected, resulting in single words being written disjointedly and multiple words written without any white spaces between them. This paper addresses the problems of word segmentation and zero-width non-joiner (ZWNJ) recognition in Persian, which we approach jointly as a sequence labeling problem. We achieved a macro-averaged F1-score of 92.40% on a carefully collected corpus of 500 sentences with a high level of difficulty.
BibTeX
@inproceedings{doostmohammadi-etal-2020-joint,
title = "Joint {P}ersian Word Segmentation Correction and Zero-Width Non-Joiner Recognition Using {BERT}",
author = "Doostmohammadi, Ehsan and
Nassajian, Minoo and
Rahimi, Adel",
editor = "Scott, Donia and
Bel, Nuria and
Zong, Chengqing",
booktitle = "Proceedings of the 28th International Conference on Computational Linguistics",
month = dec,
year = "2020",
address = "Barcelona, Spain (Online)",
publisher = "International Committee on Computational Linguistics",
url = "https://aclanthology.org/2020.coling-main.406/",
doi = "10.18653/v1/2020.coling-main.406",
pages = "4612--4618"
}