DBQR-QA: A Question Answering Dataset on a Hybrid of Database Querying and Reasoning
Rungsiman Nararatwong, Chung-Chi Chen, Natthawut Kertkeidkachorn, Hiroya Takamura, Ryutaro Ichise
Abstract
This paper introduces the Database Querying and Reasoning Dataset for Question Answering (DBQR-QA), aimed at addressing the gap in current question-answering (QA) research by emphasizing the essential processes of database querying and reasoning to answer questions. Specifically designed to accommodate sequential questions and multi-hop queries, DBQR-QA more accurately mirrors the dynamics of real-world information retrieval and analysis, with a particular focus on the financial reports of US companies. The dataset’s construction, the challenges encountered during its development, the performance of large language models on this dataset, and a human evaluation are thoroughly discussed to illustrate the dataset’s complexity and highlight future research directions in querying and reasoning tasks.
BibTeX
@inproceedings{nararatwong-etal-2024-dbqr,
title = "{DBQR}-{QA}: A Question Answering Dataset on a Hybrid of Database Querying and Reasoning",
author = "Nararatwong, Rungsiman and
Chen, Chung-Chi and
Kertkeidkachorn, Natthawut and
Takamura, Hiroya and
Ichise, Ryutaro",
editor = "Ku, Lun-Wei and
Martins, Andre and
Srikumar, Vivek",
booktitle = "Findings of the Association for Computational Linguistics: ACL 2024",
month = aug,
year = "2024",
address = "Bangkok, Thailand",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2024.findings-acl.900/",
doi = "10.18653/v1/2024.findings-acl.900",
pages = "15169--15182"
}