In Factuality: Efficient Integration of Relevant Facts for Visual Question Answering
Peter Vickers, Nikolaos Aletras, Emilio Monti, Loïc Barrault
Abstract
Visual Question Answering (VQA) methods aim at leveraging visual input to answer questions that may require complex reasoning over entities. Current models are trained on labelled data that may be insufficient to learn complex knowledge representations. In this paper, we propose a new method to enhance the reasoning capabilities of a multi-modal pretrained model (Vision+Language BERT) by integrating facts extracted from an external knowledge base. Evaluation on the KVQA dataset benchmark demonstrates that our method outperforms competitive baselines by 19%, achieving new state-of-the-art results. We also perform an extensive analysis highlighting the limitations of our best performing model through an ablation study.
BibTeX
@inproceedings{vickers-etal-2021-factuality,
title = "In Factuality: Efficient Integration of Relevant Facts for Visual Question Answering",
author = {Vickers, Peter and
Aletras, Nikolaos and
Monti, Emilio and
Barrault, Lo{\"i}c},
editor = "Zong, Chengqing and
Xia, Fei and
Li, Wenjie and
Navigli, Roberto",
booktitle = "Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)",
month = aug,
year = "2021",
address = "Online",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2021.acl-short.60/",
doi = "10.18653/v1/2021.acl-short.60",
pages = "468--475"
}