TaxReasoning: Benchmarking Knowledge-Intensive Mathematical Reasoning with Evolving Tax Laws
Nan Hu, Yike Wu, Jiaye Li, HuiKang Hu, Guilin Qi, Songlin Zhai, Yongrui Chen, Tianxing Wu
Abstract
Recent studies have explored the capabilities of large language models (LLMs) in solving knowledge-intensive mathematical reasoning problems. However, existing benchmarks predominantly involve static theorems that LLMs have encountered during pretraining, failing to assess dynamic knowledge integration. In this work, we introduce TaxReasoning, a novel benchmark designed to evaluate LLMs’ abilities in real-world tax calculation scenarios. These tasks require not only mathematical reasoning and numerical computation, but also the extraction and application of complex, frequently updated tax regulations. Through extensive experiments with state-of-the-art LLMs using diverse prompting strategies and knowledge augmentation techniques, we uncover substantial limitations in their ability to handle dynamic, knowledge-intensive questions—primarily due to missing domain-specific knowledge and ineffective retrieval. Even the best-performing models fall significantly short of human-level performance. Our analysis points to key avenues for improvement, including enhancing LLMs
BibTeX
@inproceedings{aaai2026_taxreasoningbenc,
title = {TaxReasoning: Benchmarking Knowledge-Intensive Mathematical Reasoning with Evolving Tax Laws},
author = {Nan Hu and Yike Wu and Jiaye Li and HuiKang Hu and Guilin Qi and Songlin Zhai and Yongrui Chen and Tianxing Wu and Tongtong Wu and Jiaoyan Chen and Jeff Z. Pan},
booktitle = {AAAI 2026},
year = {2026}
}