DocEE: A Large-Scale and Fine-grained Benchmark for Document-level Event Extraction
MeiHan Tong, Bin Xu, Shuai Wang, Meihuan Han, Yixin Cao, Jiangqi Zhu, Siyu Chen, Lei Hou
Abstract
Event extraction aims to identify an event and then extract the arguments participating in the event. Despite the great success in sentence-level event extraction, events are more naturally presented in the form of documents, with event arguments scattered in multiple sentences. However, a major barrier to promote document-level event extraction has been the lack of large-scale and practical training and evaluation datasets. In this paper, we present DocEE, a new document-level event extraction dataset including 27,000+ events, 180,000+ arguments. We highlight three features: large-scale manual annotations, fine-grained argument types and application-oriented settings. Experiments show that there is still a big gap between state-of-the-art models and human beings (41% Vs 85% in F1 score), indicating that DocEE is an open issue. DocEE is now available at https://github.com/tongmeihan1995/DocEE.git.
BibTeX
@inproceedings{tong-etal-2022-docee,
title = "{D}oc{EE}: A Large-Scale and Fine-grained Benchmark for Document-level Event Extraction",
author = "Tong, MeiHan and
Xu, Bin and
Wang, Shuai and
Han, Meihuan and
Cao, Yixin and
Zhu, Jiangqi and
Chen, Siyu and
Hou, Lei and
Li, Juanzi",
editor = "Carpuat, Marine and
de Marneffe, Marie-Catherine and
Meza Ruiz, Ivan Vladimir",
booktitle = "Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies",
month = jul,
year = "2022",
address = "Seattle, United States",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2022.naacl-main.291/",
doi = "10.18653/v1/2022.naacl-main.291",
pages = "3970--3982"
}