Benchmarking Visual LLMs Resilience to Unanswerable Questions on Visually Rich Documents
Davide Napolitano, Luca Cagliero, Fabrizio Battiloro
Abstract
The evolution of Visual Large Language Models (VLLMs) has revolutionized the automatic understanding of Visually Rich Documents (VRDs), which contain both textual and visual elements. Although VLLMs excel in Visual Question Answering (VQA) on multi-page VRDs, their ability to detect unanswerable questions is still an open research question. Our research delves into the robustness of the VLLMs to plausible yet unanswerable questions, i.e., questions that appear valid but cannot be answered due to subtle corruptions caused by swaps between related concepts or plausible question formulations. Corruptions are generated by replacing the original natural language entities with other ones of the same type, belonging to different document elements, and in different layout positions or pages of the related document. To this end, we present VRD-UQA (VISUALLY RICH DOCUMENT UNANSWERABLE QUESTION ANSWERING), a benchmark for evaluating VLLMs
BibTeX
@inproceedings{aaai2026_benchmarkingvisu,
title = {Benchmarking Visual LLMs Resilience to Unanswerable Questions on Visually Rich Documents},
author = {Davide Napolitano and Luca Cagliero and Fabrizio Battiloro},
booktitle = {AAAI 2026},
year = {2026}
}