Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CompileAgent: Automated Real-World Repo-Level Compilation with Tool-Integrated LLM-based Agent System

Li Hu, Guoqiang Chen, Xiuwei Shang, Shaoyin Cheng, Benlong Wu, Gangyang Li, Xu Zhu, Weiming Zhang, Nenghai Yu

TL;DR

The paper tackles the challenge of automated repo-level compilation by introducing CompileAgent, an LLM-based agent framework that uses a flow-based strategy and five specialized tools to locate compilation instructions and resolve build errors. It formalizes the repo-level compilation task, builds the CompileAgentBench benchmark, and demonstrates substantial improvements in compilation success across seven LLMs, along with reductions in time and cost. Key contributions include the two core modules (CompileNavigator and ErrorSolver), the tool suite, and a rigorous ablation and failure analysis that support the approach's practicality. The work demonstrates strong potential for real-world automation of complex software builds and scalable cross-language, cross-architecture compilation tasks.

Abstract

With open-source projects growing in size and complexity, manual compilation becomes tedious and error-prone, highlighting the need for automation to improve efficiency and accuracy. However, the complexity of compilation instruction search and error resolution makes automatic compilation challenging. Inspired by the success of LLM-based agents in various fields, we propose CompileAgent, the first LLM-based agent framework dedicated to repo-level compilation. CompileAgent integrates five tools and a flow-based agent strategy, enabling interaction with software artifacts for compilation instruction search and error resolution. To measure the effectiveness of our method, we design a public repo-level benchmark CompileAgentBench, and we also design two baselines for comparison by combining two compilation-friendly schemes. The performance on this benchmark shows that our method significantly improves the compilation success rate, ranging from 10% to 71%. Meanwhile, we evaluate the performance of CompileAgent under different agent strategies and verify the effectiveness of the flow-based strategy. Additionally, we emphasize the scalability of CompileAgent, further expanding its application prospects.

CompileAgent: Automated Real-World Repo-Level Compilation with Tool-Integrated LLM-based Agent System

TL;DR

The paper tackles the challenge of automated repo-level compilation by introducing CompileAgent, an LLM-based agent framework that uses a flow-based strategy and five specialized tools to locate compilation instructions and resolve build errors. It formalizes the repo-level compilation task, builds the CompileAgentBench benchmark, and demonstrates substantial improvements in compilation success across seven LLMs, along with reductions in time and cost. Key contributions include the two core modules (CompileNavigator and ErrorSolver), the tool suite, and a rigorous ablation and failure analysis that support the approach's practicality. The work demonstrates strong potential for real-world automation of complex software builds and scalable cross-language, cross-architecture compilation tasks.

Abstract

With open-source projects growing in size and complexity, manual compilation becomes tedious and error-prone, highlighting the need for automation to improve efficiency and accuracy. However, the complexity of compilation instruction search and error resolution makes automatic compilation challenging. Inspired by the success of LLM-based agents in various fields, we propose CompileAgent, the first LLM-based agent framework dedicated to repo-level compilation. CompileAgent integrates five tools and a flow-based agent strategy, enabling interaction with software artifacts for compilation instruction search and error resolution. To measure the effectiveness of our method, we design a public repo-level benchmark CompileAgentBench, and we also design two baselines for comparison by combining two compilation-friendly schemes. The performance on this benchmark shows that our method significantly improves the compilation success rate, ranging from 10% to 71%. Meanwhile, we evaluate the performance of CompileAgent under different agent strategies and verify the effectiveness of the flow-based strategy. Additionally, we emphasize the scalability of CompileAgent, further expanding its application prospects.
Paper Structure (23 sections, 4 figures, 4 tables)

This paper contains 23 sections, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Background
  3. LLMs and Agents
  4. Automatic Compilation
  5. Repo-Level Compilation Task
  6. Method
  7. Designed Module
  8. CompileNavigator
  9. ErrorSolver
  10. Agent Strategy
  11. Experiment
  12. Experimental Setup
  13. Repo-Level Compilation Performance
  14. Strategy Performance
  15. Ablation Study
  16. ...and 8 more sections

Figures (4)

  • Figure 1: An illustrative example of the automated repo-level compilation. The task input contains code repository documentation and structure, and the automated compilation system can interact with the interactive environment.
  • Figure 2: The overview of CompileAgent. By providing the repository of a given project, the automated compilation process can be seamlessly completed using the designed modules and agent strategy. Agents not explicitly specified are driven by DeepSeek-v2.5.
  • Figure 3: The details of Readme-AI.
  • Figure 4: The details of RAG.