SemRel2024: A Collection of Semantic Textual Relatedness Datasets for 13 Languages
Nedjma Ousidhoum, Shamsuddeen Muhammad, Mohamed Abdalla, Idris Abdulmumin, Ibrahim Ahmad, Sanchit Ahuja, Alham Aji, Vladimir Araujo
Abstract
Exploring and quantifying semantic relatedness is central to representing language and holds significant implications across various NLP tasks. While earlier NLP research primarily focused on semantic similarity, often within the English language context, we instead investigate the broader phenomenon of semantic relatedness. In this paper, we present SemRel, a new semantic relatedness dataset collection annotated by native speakers across 13 languages: Afrikaans, Algerian Arabic, Amharic, English, Hausa, Hindi, Indonesian, Kinyarwanda, Marathi, Moroccan Arabic, Modern Standard Arabic, Spanish, and Telugu. These languages originate from five distinct language families and are predominantly spoken in Africa and Asia – regions characterised by a relatively limited availability of NLP resources. Each instance in the SemRel datasets is a sentence pair associated with a score that represents the degree of semantic textual relatedness between the two sentences. The scores are obtained using a comparative annotation framework. We describe the data collection and annotation processes, challenges when building the datasets, baseline experiments, and their impact and utility in NLP.
BibTeX
@inproceedings{ousidhoum-etal-2024-semrel2024,
title = "{S}em{R}el2024: A Collection of Semantic Textual Relatedness Datasets for 13 Languages",
author = "Ousidhoum, Nedjma and
Muhammad, Shamsuddeen and
Abdalla, Mohamed and
Abdulmumin, Idris and
Ahmad, Ibrahim and
Ahuja, Sanchit and
Aji, Alham and
Araujo, Vladimir and
Ayele, Abinew and
Baswani, Pavan and
Beloucif, Meriem and
Biemann, Chris and
Bourhim, Sofia and
Kock, Christine and
Dekebo, Genet and
Hourrane, Oumaima and
Kanumolu, Gopichand and
Madasu, Lokesh and
Rutunda, Samuel and
Shrivastava, Manish and
Solorio, Thamar and
Surange, Nirmal and
Tilaye, Hailegnaw and
Vishnubhotla, Krishnapriya and
Winata, Genta and
Yimam, Seid and
Mohammad, Saif",
editor = "Ku, Lun-Wei and
Martins, Andre and
Srikumar, Vivek",
booktitle = "Findings of the Association for Computational Linguistics: ACL 2024",
month = aug,
year = "2024",
address = "Bangkok, Thailand",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2024.findings-acl.147/",
doi = "10.18653/v1/2024.findings-acl.147",
pages = "2512--2530"
}