Document Understanding Dataset and Evaluation (DUDE)
Jordy Van Landeghem, Rubèn Tito, Łukasz Borchmann, Michał Pietruszka, Pawel Joziak, Rafal Powalski, Dawid Jurkiewicz, Mickael Coustaty
Abstract
We call on the Document AI (DocAI) community to re-evaluate current methodologies and embrace the challenge of creating more practically-oriented benchmarks. Document Understanding Dataset and Evaluation (DUDE) seeks to remediate the halted research progress in understanding visually-rich documents (VRDs). We present a new dataset with novelties related to types of questions, answers, and document layouts based on multi-industry, multi-domain, and multi-page VRDs of various origins and dates. Moreover, we are pushing the boundaries of current methods by creating multi-task and multi-domain evaluation setups that more accurately simulate real-world situations where powerful generalization and adaptation under low-resource settings are desired. DUDE aims to set a new standard as a more practical, long-standing benchmark for the community, and we hope that it will lead to future extensions and contributions that address real-world challenges. Finally, our work illustrates the importance of finding more efficient ways to model language, images, and layout in DocAI.
BibTeX
@inproceedings{iccv2023_documentundersta,
title = {Document Understanding Dataset and Evaluation (DUDE)},
author = {Jordy Van Landeghem and Rubèn Tito and Łukasz Borchmann and Michał Pietruszka and Pawel Joziak and Rafal Powalski and Dawid Jurkiewicz and Mickael Coustaty and Bertrand Anckaert and Ernest Valveny and Matthew Blaschko and Sien Moens and Tomasz Stanislawek},
booktitle = {ICCV 2023},
year = {2023}
}