LongTableBench: Benchmarking Long-Context Table Reasoning across Real-World Formats and Domains
Liyao Li, Jiaming Tian, Hao Chen, Wentao Ye, Chao Ye, Haobo Wang, Ningtao Wang, Xing Fu
Abstract
We introduce LongTableBench , a benchmark for evaluating long-context reasoning over semi-structured tables across diverse formats, tasks, and domains. It comprises 5,950 QA instances spanning 7 table formats (e.g., Markdown, HTML, SQL), 18 domains, and input lengths up to 128K tokens, including multi-turn and multi-table settings. To ensure data quality, we combine symbolic supervision, cross-model validation, and human review. Evaluating 52 LLMs—including general-purpose, table-specific, and reasoning-enhanced models—reveals that only the strongest models maintain robust performance under increasing context lengths and format diversity. We further show that end-to-end models outperform compression-based approaches, especially on tasks requiring semantic integration. LongTableBench provides a rigorous, scalable testbed for advancing long-context tabular understanding and highlights key limitations in current LLMs’ structural and reasoning capabilities. The code and data are available at https://github.com/liyaooi/LongTableBench .
BibTeX
@inproceedings{emnlp2025_longtablebenchbe,
title = {LongTableBench: Benchmarking Long-Context Table Reasoning across Real-World Formats and Domains},
author = {Liyao Li and Jiaming Tian and Hao Chen and Wentao Ye and Chao Ye and Haobo Wang and Ningtao Wang and Xing Fu and Gang Chen and Junbo Zhao},
booktitle = {EMNLP 2025},
year = {2025}
}