Experimental Standards for Deep Learning in Natural Language Processing Research
Dennis Ulmer, Elisa Bassignana, Max Müller-Eberstein, Daniel Varab, Mike Zhang, Rob van der Goot, Christian Hardmeier, Barbara Plank
Abstract
The field of Deep Learning (DL) has undergone explosive growth during the last decade, with a substantial impact on Natural Language Processing (NLP) as well. Yet, compared to more established disciplines, a lack of common experimental standards remains an open challenge to the field at large. Starting from fundamental scientific principles, we distill ongoing discussions on experimental standards in NLP into a single, widely-applicable methodology. Following these best practices is crucial to strengthen experimental evidence, improve reproducibility and enable scientific progress. These standards are further collected in a public repository to help them transparently adapt to future needs.
BibTeX
@inproceedings{ulmer-etal-2022-experimental,
title = "Experimental Standards for Deep Learning in Natural Language Processing Research",
author = {Ulmer, Dennis and
Bassignana, Elisa and
M{\"u}ller-Eberstein, Max and
Varab, Daniel and
Zhang, Mike and
van der Goot, Rob and
Hardmeier, Christian and
Plank, Barbara},
editor = "Goldberg, Yoav and
Kozareva, Zornitsa and
Zhang, Yue",
booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2022",
month = dec,
year = "2022",
address = "Abu Dhabi, United Arab Emirates",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2022.findings-emnlp.196/",
doi = "10.18653/v1/2022.findings-emnlp.196",
pages = "2673--2692"
}