CompileAgent: Automated Real-World Repo-Level Compilation with Tool-Integrated LLM-based Agent System
Li Hu, Guoqiang Chen, Xiuwei Shang, Shaoyin Cheng, Benlong Wu, LiGangyang LiGangyang, Xu Zhu, Weiming Zhang
Abstract
With open-source projects growing in size and complexity, manual compilation becomes tedious and error-prone, highlighting the need for automation to improve efficiency and accuracy. However, the complexity of compilation instruction search and error resolution makes automatic compilation challenging. Inspired by the success of LLM-based agents in various fields, we propose CompileAgent, the first LLM-based agent framework dedicated to repo-level compilation. CompileAgent integrates five tools and a flow-based agent strategy, enabling interaction with software artifacts for compilation instruction search and error resolution. To measure the effectiveness of our method, we design a public repo-level benchmark CompileAgentBench, and we also design two baselines for comparison by combining two compilation-friendly schemes. The performance on this benchmark shows that our method significantly improves the compilation success rate, ranging from 10% to 71%. Meanwhile, we evaluate the performance of CompileAgent under different agent strategies and verify the effectiveness of the flow-based strategy. Additionally, we emphasize the scalability of CompileAgent, further expanding its application prospects. The complete code and data are available at https://github.com/Ch3nYe/AutoCompiler.
BibTeX
@inproceedings{hu-etal-2025-compileagent,
title = "{C}ompile{A}gent: Automated Real-World Repo-Level Compilation with Tool-Integrated {LLM}-based Agent System",
author = "Hu, Li and
Chen, Guoqiang and
Shang, Xiuwei and
Cheng, Shaoyin and
Wu, Benlong and
LiGangyang, LiGangyang and
Zhu, Xu and
Zhang, Weiming and
Yu, Nenghai",
editor = "Che, Wanxiang and
Nabende, Joyce and
Shutova, Ekaterina and
Pilehvar, Mohammad Taher",
booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
month = jul,
year = "2025",
address = "Vienna, Austria",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.acl-long.103/",
doi = "10.18653/v1/2025.acl-long.103",
pages = "2078--2091",
ISBN = "979-8-89176-251-0"
}