SportReason: Evaluating Retrieval-Augmented Reasoning across Tables and Text for Sports Question Answering
Kaiyue Feng, Siyue Zhang, Bingsen Chen, Yilun Zhao, Chen Zhao
Abstract
We present SportReason, a benchmark for retrieval-augmented reasoning on numerical sports questions. Unlike existing benchmarks limited to one or two evidence units, SportReason requires combining and reasoning across free-text, structured tables, and semi-structured infoboxes. We provide 3,000 human-verified QA pairs by repurposing existing QA and table generation datasets, and by prompting large language models (LLMs). Each pair is grounded in multiple evidence from a multi-modal Wikipedia corpus containing 200K knowledge contexts. We evaluate existing retrievers and rerankers, along with agentic Retrieval-Augmented Generation (RAG) systems. The experimental results show that multi-evidence retrieval remains a challenge. Agentic RAG systems (e.g., Search-o1), despite iterative retrieval and reasoning capabilities, fail to improve performance due to imprecise queries, simple training, and distracting information.
BibTeX
@inproceedings{emnlp2025_sportreasonevalu,
title = {SportReason: Evaluating Retrieval-Augmented Reasoning across Tables and Text for Sports Question Answering},
author = {Kaiyue Feng and Siyue Zhang and Bingsen Chen and Yilun Zhao and Chen Zhao},
booktitle = {EMNLP 2025},
year = {2025}
}