FinSearchComp: Towards a Realistic, Expert-Level Evaluation of Financial Search and Reasoning
LIANG HU, Jianpeng Jiao, Jiashuo Liu, Dongyuan Mutu, Yanle Ren, Zhoufutu Wen, Kaiyuan Zhang, Xuanliang Zhang
Abstract
Search has emerged as core infrastructure for LLM-based agents and is widely viewed as critical on the path toward more general intelligence. Finance is a particularly demanding proving ground: analysts routinely conduct complex, multi-step searches over time-sensitive, domain-specific data, making it ideal for assessing both search proficiency and knowledge-grounded reasoning. Yet no existing open financial datasets evaluate data searching capability of end-to-end agents, largely because constructing realistic, complicated tasks requires deep financial expertise and time-sensitive data is hard to evaluate. We present FinSearchComp, the first fully open-source agent benchmark for realistic, open-domain financial search and reasoning. FinSearchComp comprises three tasks, Time-Sensitive Data Fetching, Simple Historical Lookup, and Complex Historical Investigation, closely reproducing real-world financial analyst workflows. To ensure difficulty and reliability, we engage $70$ professional financial experts for annotation and implement a rigorous multi-stage quality-assurance pipeline. The benchmark includes $635$ questions spanning global and Greater China markets, and we evaluate $21$ models (products) on it. Grok 4 (web) tops the global subset, approaching expert-level accuracy. DouBao (web) leads on the Greater China subset. Experimental analyses show that equipping agents with web search and financial plugins substantially improves results on FinSearchComp, and the country origin of models and tools impact performance significantly. By aligning with realistic analyst tasks and providing end-to-end evaluation, FinSearchComp offers a professional, high-difficulty testbed for complex financial search and reasoning.
BibTeX
@inproceedings{
hu2026finsearchcomp,
title={FinSearchComp: Towards a Realistic, Expert-Level Evaluation of Financial Search and Reasoning},
author={LIANG HU and Jianpeng Jiao and Jiashuo Liu and Dongyuan Mutu and Yanle Ren and Zhoufutu Wen and Kaiyuan Zhang and Xuanliang Zhang and Xiang Gao and Tianci He and FEI HU and Yali Liao and Zaiyuan Wang and Jingkai Liu and Sun Daibin and Ziqing Zeng and Zhiyuan Zeng and Chenghao Yang and Qianyu Yang and Mingren Yin and Ge Zhang and Xinyi zhang and Xiying ZHAO and Zhu Zhenwei and Hongseok Namkoong and Wenhao Huang},
booktitle={The Fourteenth International Conference on Learning Representations},
year={2026},
url={https://openreview.net/forum?id=8AJbbbe2ni}
}