EMNLP 2022finding35 citations
SlovakBERT: Slovak Masked Language Model
Matúš Pikuliak, Štefan Grivalský, Martin Konôpka, Miroslav Blšták, Martin Tamajka, Viktor Bachratý, Marian Simko, Pavol Balážik
Abstract
We introduce a new Slovak masked language model called SlovakBERT. This is to our best knowledge the first paper discussing Slovak transformers-based language models. We evaluate our model on several NLP tasks and achieve state-of-the-art results. This evaluation is likewise the first attempt to establish a benchmark for Slovak language models. We publish the masked language model, as well as the fine-tuned models for part-of-speech tagging, sentiment analysis and semantic textual similarity.
BibTeX
@inproceedings{pikuliak-etal-2022-slovakbert,
title = "{S}lovak{BERT}: {S}lovak Masked Language Model",
author = "Pikuliak, Mat{\'u}{\v{s}} and
Grivalsk{\'y}, {\v{S}}tefan and
Kon{\^o}pka, Martin and
Bl{\v{s}}t{\'a}k, Miroslav and
Tamajka, Martin and
Bachrat{\'y}, Viktor and
Simko, Marian and
Bal{\'a}{\v{z}}ik, Pavol and
Trnka, Michal and
Uhl{\'a}rik, Filip",
editor = "Goldberg, Yoav and
Kozareva, Zornitsa and
Zhang, Yue",
booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2022",
month = dec,
year = "2022",
address = "Abu Dhabi, United Arab Emirates",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2022.findings-emnlp.530/",
doi = "10.18653/v1/2022.findings-emnlp.530",
pages = "7156--7168"
}