Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Iterative Experience Refinement of Software-Developing Agents

Chen Qian, Jiahao Li, Yufan Dang, Wei Liu, YiFei Wang, Zihao Xie, Weize Chen, Cheng Yang, Yingli Zhang, Zhiyuan Liu, Maosong Sun

TL;DR

This work tackles the rigidity of static, heuristic experiences in LLM-based software-developing agents by introducing Iterative Experience Refinement (IER). IER enables continual acquisition, utilization, and elimination of experiences across task batches, supported by two propagation patterns: successive (latest-predecessor) and cumulative (all-history). A heuristic elimination mechanism curbs unbounded growth of the experience space, preserving high-quality, frequently-used experiences. Empirical results on the SRDD dataset show that while the successive pattern can yield higher peaks in performance, the cumulative pattern offers more stable improvements, and elimination enables strong performance with only a small, high-quality subset of experiences. These findings underscore the value of dynamic experience refinement for scalable, autonomous software development.

Abstract

Autonomous agents powered by large language models (LLMs) show significant potential for achieving high autonomy in various scenarios such as software development. Recent research has shown that LLM agents can leverage past experiences to reduce errors and enhance efficiency. However, the static experience paradigm, reliant on a fixed collection of past experiences acquired heuristically, lacks iterative refinement and thus hampers agents' adaptability. In this paper, we introduce the Iterative Experience Refinement framework, enabling LLM agents to refine experiences iteratively during task execution. We propose two fundamental patterns: the successive pattern, refining based on nearest experiences within a task batch, and the cumulative pattern, acquiring experiences across all previous task batches. Augmented with our heuristic experience elimination, the method prioritizes high-quality and frequently-used experiences, effectively managing the experience space and enhancing efficiency. Extensive experiments show that while the successive pattern may yield superior results, the cumulative pattern provides more stable performance. Moreover, experience elimination facilitates achieving better performance using just 11.54% of a high-quality subset.

Iterative Experience Refinement of Software-Developing Agents

TL;DR

This work tackles the rigidity of static, heuristic experiences in LLM-based software-developing agents by introducing Iterative Experience Refinement (IER). IER enables continual acquisition, utilization, and elimination of experiences across task batches, supported by two propagation patterns: successive (latest-predecessor) and cumulative (all-history). A heuristic elimination mechanism curbs unbounded growth of the experience space, preserving high-quality, frequently-used experiences. Empirical results on the SRDD dataset show that while the successive pattern can yield higher peaks in performance, the cumulative pattern offers more stable improvements, and elimination enables strong performance with only a small, high-quality subset of experiences. These findings underscore the value of dynamic experience refinement for scalable, autonomous software development.

Abstract

Autonomous agents powered by large language models (LLMs) show significant potential for achieving high autonomy in various scenarios such as software development. Recent research has shown that LLM agents can leverage past experiences to reduce errors and enhance efficiency. However, the static experience paradigm, reliant on a fixed collection of past experiences acquired heuristically, lacks iterative refinement and thus hampers agents' adaptability. In this paper, we introduce the Iterative Experience Refinement framework, enabling LLM agents to refine experiences iteratively during task execution. We propose two fundamental patterns: the successive pattern, refining based on nearest experiences within a task batch, and the cumulative pattern, acquiring experiences across all previous task batches. Augmented with our heuristic experience elimination, the method prioritizes high-quality and frequently-used experiences, effectively managing the experience space and enhancing efficiency. Extensive experiments show that while the successive pattern may yield superior results, the cumulative pattern provides more stable performance. Moreover, experience elimination facilitates achieving better performance using just 11.54% of a high-quality subset.
Paper Structure (22 sections, 7 equations, 7 figures, 1 table)

This paper contains 22 sections, 7 equations, 7 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Experience Acquisition and Utilization
  5. Acquisition
  6. Utilization
  7. Experience Propagation
  8. Successive Pattern
  9. Cumulative Pattern
  10. Experience Elimination
  11. Evaluation
  12. Baselines
  13. Datasets
  14. Metrics
  15. Implementation Details
  16. ...and 7 more sections

Figures (7)

  • Figure 1: The task execution chain constructed for shortcut-oriented experience acquiring. The execution chain creates procedural trajectories for various training tasks, where we acquire "shortcuts" linking non-adjacent nodes as agents' experiences.
  • Figure 2: The successive pattern (left) allows each task batch to utilize the experience pool collected from the preceding batch. The cumulative pattern (right) enables each batch of tasks to leverage the experience pool acquired from all previous batches.
  • Figure 3: The average performance for each task batch across various dimensions.
  • Figure 4: The phase efficiency per task batch across various dimensions. Review Efficiency is calculated by averaging the rounds of code review, derived from the difference between the actual and maximum review rounds conducted by agents. Test Efficiency measures efficiency during testing, while Overall Efficiency accounts for all interactive rounds across phases, reflecting agents' whole-process software optimization. Higher results indicate faster adherence to software standards, reducing the necessity for additional manual involvement and thereby enhancing software generation efficiency.
  • Figure 5: The retrieval hit ratio across different task batches, calculated by dividing the number of experiences retrieved by the total number of experiences.
  • ...and 2 more figures