A Survey on Out-of-Distribution Evaluation of Neural NLP Models

Xinzhe Li; Ming Liu; Shang Gao; Wray Buntine

IJCAI 2023poster23 citations

A Survey on Out-of-Distribution Evaluation of Neural NLP Models

Xinzhe Li, Ming Liu, Shang Gao, Wray Buntine

Abstract

Adversarial robustness, domain generalization and dataset biases are three active lines of research contributing to out-of-distribution (OOD) evaluation on neural NLP models. However, a comprehensive, integrated discussion of the three research lines is still lacking in the literature. This survey will 1) compare the three lines of research under a unifying definition; 2) summarize their data-generating processes and evaluation protocols for each line of research; and 3) emphasize the challenges and opportunities for future work.

Survey: Natural Language ProcessingSurvey: Machine LearningSurvey: Uncertainty in AISurvey: AI Ethics, Trust, Fairness

BibTeX

@inproceedings{ijcai2023p749,
  title     = {A Survey on Out-of-Distribution Evaluation of Neural NLP Models},
  author    = {Li, Xinzhe and Liu, Ming and Gao, Shang and Buntine, Wray},
  booktitle = {Proceedings of the Thirty-Second International Joint Conference on
               Artificial Intelligence, {IJCAI-23}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {Edith Elkind},
  pages     = {6683--6691},
  year      = {2023},
  month     = {8},
  note      = {Survey Track},
  doi       = {10.24963/ijcai.2023/749},
  url       = {https://doi.org/10.24963/ijcai.2023/749},
}