Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

EvoConfig: Self-Evolving Multi-Agent Systems for Efficient Autonomous Environment Configuration

Xinshuai Guo, Jiayi Kuang, Linyue Pan, Yinghui Li, Yangning Li, Hai-Tao Zheng, Ying Shen, Di Yin, Xing Sun

TL;DR

EvoConfig tackles the challenge of reliably building executable runtime environments for large-scale SWE tasks by introducing a self-evolving, multi-agent framework. It decouples execution, diagnosis, and repair into three components: an environment info extractor, a main configuration agent using streaming ReAct reasoning, and an expert diagnosis agent that analyzes outcomes, potentially creates diagnostic tools, and adapts its rules over time. Empirical results show EvoConfig matching or exceeding Repo2Run on a large repository set and achieving strong gains on harder EnvBench tasks, along with superior process-level error perception and repair guidance. The work demonstrates that adaptive, fine-grained diagnosis integrated with environment construction can substantially boost robustness and efficiency in automated environment synthesis, with practical implications for scalable SWE agent deployment and benchmarking.

Abstract

A reliable executable environment is the foundation for ensuring that large language models solve software engineering tasks. Due to the complex and tedious construction process, large-scale configuration is relatively inefficient. However, most methods always overlook fine-grained analysis of the actions performed by the agent, making it difficult to handle complex errors and resulting in configuration failures. To address this bottleneck, we propose EvoConfig, an efficient environment configuration framework that optimizes multi-agent collaboration to build correct runtime environments. EvoConfig features an expert diagnosis module for fine-grained post-execution analysis, and a self-evolving mechanism that lets expert agents self-feedback and dynamically adjust error-fixing priorities in real time. Empirically, EvoConfig matches the previous state-of-the-art Repo2Run on Repo2Run's 420 repositories, while delivering clear gains on harder cases: on the more challenging Envbench, EvoConfig achieves a 78.1% success rate, outperforming Repo2Run by 7.1%. Beyond end-to-end success, EvoConfig also demonstrates stronger debugging competence, achieving higher accuracy in error identification and producing more effective repair recommendations than existing methods.

EvoConfig: Self-Evolving Multi-Agent Systems for Efficient Autonomous Environment Configuration

TL;DR

EvoConfig tackles the challenge of reliably building executable runtime environments for large-scale SWE tasks by introducing a self-evolving, multi-agent framework. It decouples execution, diagnosis, and repair into three components: an environment info extractor, a main configuration agent using streaming ReAct reasoning, and an expert diagnosis agent that analyzes outcomes, potentially creates diagnostic tools, and adapts its rules over time. Empirical results show EvoConfig matching or exceeding Repo2Run on a large repository set and achieving strong gains on harder EnvBench tasks, along with superior process-level error perception and repair guidance. The work demonstrates that adaptive, fine-grained diagnosis integrated with environment construction can substantially boost robustness and efficiency in automated environment synthesis, with practical implications for scalable SWE agent deployment and benchmarking.

Abstract

A reliable executable environment is the foundation for ensuring that large language models solve software engineering tasks. Due to the complex and tedious construction process, large-scale configuration is relatively inefficient. However, most methods always overlook fine-grained analysis of the actions performed by the agent, making it difficult to handle complex errors and resulting in configuration failures. To address this bottleneck, we propose EvoConfig, an efficient environment configuration framework that optimizes multi-agent collaboration to build correct runtime environments. EvoConfig features an expert diagnosis module for fine-grained post-execution analysis, and a self-evolving mechanism that lets expert agents self-feedback and dynamically adjust error-fixing priorities in real time. Empirically, EvoConfig matches the previous state-of-the-art Repo2Run on Repo2Run's 420 repositories, while delivering clear gains on harder cases: on the more challenging Envbench, EvoConfig achieves a 78.1% success rate, outperforming Repo2Run by 7.1%. Beyond end-to-end success, EvoConfig also demonstrates stronger debugging competence, achieving higher accuracy in error identification and producing more effective repair recommendations than existing methods.
Paper Structure (39 sections, 3 equations, 4 figures, 8 tables)

This paper contains 39 sections, 3 equations, 4 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Formulation
  3. Task Definition
  4. Iterative Configuration Process
  5. Method
  6. Environment Info Extraction Module
  7. Dependency Management Strategy.
  8. Project Importability.
  9. Test Structure Hypothesis.
  10. Main Environment Configuration Module
  11. Self-Evolving Expert Diagnosis Module
  12. Experimental Setup
  13. Environment Build Success Evaluation
  14. Dataset and Baselines.
  15. Evaluation Metrics.
  16. ...and 24 more sections

Figures (4)

  • Figure 1: Self-Evolving Diagnostic Process.
  • Figure 2: Workflow of EvoConfig. A main configuration agent performs interactive environment setup, while self-evolving expert diagnostic agent analyzes execution feedback and provide adaptive guidance. The validated command sequence is consolidated into a runnable Dockerfile.
  • Figure 3: Runtime distribution of successful environment builds in the ablation study.
  • Figure 4: Ablation results of EvoConfig on process-level error correction.