Are Large Language Models Good at Lexical Semantics? A Case of Taxonomy Learning
Viktor Moskvoretskii, Alexander Panchenko, Irina Nikishina
Abstract
Recent studies on LLMs do not pay enough attention to linguistic and lexical semantic tasks, such as taxonomy learning. In this paper, we explore the capacities of Large Language Models featuring LLaMA-2 and Mistral for several Taxonomy-related tasks. We introduce a new methodology and algorithm for data collection via stochastic graph traversal leading to controllable data collection. Collected cases provide the ability to form nearly any type of graph operation. We test the collected dataset for learning taxonomy structure based on English WordNet and compare different input templates for fine-tuning LLMs. Moreover, we apply the fine-tuned models on such datasets on the downstream tasks achieving state-of-the-art results on the TexEval-2 dataset.
BibTeX
@inproceedings{moskvoretskii-etal-2024-large,
title = "Are Large Language Models Good at Lexical Semantics? A Case of Taxonomy Learning",
author = "Moskvoretskii, Viktor and
Panchenko, Alexander and
Nikishina, Irina",
editor = "Calzolari, Nicoletta and
Kan, Min-Yen and
Hoste, Veronique and
Lenci, Alessandro and
Sakti, Sakriani and
Xue, Nianwen",
booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)",
month = may,
year = "2024",
address = "Torino, Italia",
publisher = "ELRA and ICCL",
url = "https://aclanthology.org/2024.lrec-main.133/",
pages = "1498--1510"
}