IndicXNLI: Evaluating Multilingual Inference for Indian Languages
Divyanshu Aggarwal, Vivek Gupta, Anoop Kunchukuttan
Abstract
While Indic NLP has made rapid advances recently in terms of the availability of corpora and pre-trained models, benchmark datasets on standard NLU tasks are limited. To this end, we introduce INDICXNLI, an NLI dataset for 11 Indic languages. It has been created by high-quality machine translation of the original English XNLI dataset and our analysis attests to the quality of INDICXNLI. By finetuning different pre-trained LMs on this INDICXNLI, we analyze various cross-lingual transfer techniques with respect to the impact of the choice of language models, languages, multi-linguality, mix-language input, etc. These experiments provide us with useful insights into the behaviour of pre-trained models for a diverse set of languages.
BibTeX
@inproceedings{aggarwal-etal-2022-indicxnli,
title = "{I}ndic{XNLI}: Evaluating Multilingual Inference for {I}ndian Languages",
author = "Aggarwal, Divyanshu and
Gupta, Vivek and
Kunchukuttan, Anoop",
editor = "Goldberg, Yoav and
Kozareva, Zornitsa and
Zhang, Yue",
booktitle = "Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing",
month = dec,
year = "2022",
address = "Abu Dhabi, United Arab Emirates",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2022.emnlp-main.755/",
doi = "10.18653/v1/2022.emnlp-main.755",
pages = "10994--11006"
}