Thinker: Training LLMs in Hierarchical Thinking for Deep Search via Multi-Turn Interaction
Jun Xu, Xinkai Du, Yu Ao, Peilong Zhao, Yang Li, Ling Zhong, Lin Yuan, Zhongpu Bo
Abstract
Efficient retrieval of external knowledge bases and web pages is crucial for enhancing the reasoning abilities of LLMs. Previous works on training LLMs to leverage external retrievers for solving complex problems have predominantly employed end-to-end reinforcement learning. However, these approaches neglect supervision over the reasoning process, making it difficult to guarantee logical coherence and rigor. To address these limitations, we propose Thinker, a hierarchical thinking model for deep search through multi-turn interaction, making the reasoning process supervisable and verifiable. It decomposes complex problems into independently solvable sub-problems, each dually represented in both natural language and an equivalent logical function to support knowledge base and web searches. Concurrently, dependencies between sub-problems are passed as parameters via these logical functions, enhancing the logical coherence of the problem-solving process. To avoid unnecessary external searches, we perform knowledge boundary determination to check if a sub-problem is within the LLM
BibTeX
@inproceedings{aaai2026_thinkertrainingl,
title = {Thinker: Training LLMs in Hierarchical Thinking for Deep Search via Multi-Turn Interaction},
author = {Jun Xu and Xinkai Du and Yu Ao and Peilong Zhao and Yang Li and Ling Zhong and Lin Yuan and Zhongpu Bo and Xiaorui Wang and Mengshu Sun and Zhengke Gui and Dalong Zhang and Zhaoyang Wang and Wang Qiwei and Yangyang Hou and Zhiying Yin and Haofen Wang and Huajun Chen and Lei Liang and Jun Zhou},
booktitle = {AAAI 2026},
year = {2026}
}