WebWalker: Benchmarking LLMs in Web Traversal
Jialong Wu, Wenbiao Yin, Yong Jiang, Zhenglin Wang, Zekun Xi, Runnan Fang, Linhai Zhang, Yulan He
Abstract
Retrieval-augmented generation (RAG) demonstrates remarkable performance across tasks in open-domain question-answering. However, traditional search engines may retrieve shallow content, limiting the ability of LLMs to handle complex, multi-layered information. To address this, we introduce WebWalkerQA, a benchmark designed to assess the ability of LLMs to perform web traversal. It evaluates the capacity of LLMs to traverse a website’s subpages to extract high-quality data systematically. We propose WebWalker, which is a multi-agent framework that mimics human-like web navigation through an explore-critic paradigm. Extensive experimental results show that WebWalkerQA is challenging and demonstrates the effectiveness of RAG combined with WebWalker, through this horizontal and vertical integration in real-world scenarios.
BibTeX
@inproceedings{wu-etal-2025-webwalker,
title = "{W}eb{W}alker: Benchmarking {LLM}s in Web Traversal",
author = "Wu, Jialong and
Yin, Wenbiao and
Jiang, Yong and
Wang, Zhenglin and
Xi, Zekun and
Fang, Runnan and
Zhang, Linhai and
He, Yulan and
Zhou, Deyu and
Xie, Pengjun and
Huang, Fei",
editor = "Che, Wanxiang and
Nabende, Joyce and
Shutova, Ekaterina and
Pilehvar, Mohammad Taher",
booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
month = jul,
year = "2025",
address = "Vienna, Austria",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.acl-long.508/",
doi = "10.18653/v1/2025.acl-long.508",
pages = "10290--10305",
ISBN = "979-8-89176-251-0"
}