AEGIS: An Agent-based Framework for General Bug Reproduction from Issue Descriptions

TL;DR

Considering the superior performance of agent-based methods in code intelligence tasks, this paper proposes an Automated gEneral buG reproductIon Scripts generation framework, named AEGIS, which is the first agent-based framework for the task.

Abstract

In software maintenance, bug reproduction is essential for effective fault localization and repair. Manually writing reproduction scripts is a time-consuming task with high requirements for developers. Hence, automation of bug reproduction has increasingly attracted attention from researchers and practitioners. However, the existing studies on bug reproduction are generally limited to specific bug types such as program crashes, and hard to be applied to general bug reproduction. In this paper, considering the superior performance of agent-based methods in code intelligence tasks, we focus on designing an agent-based framework for the task. Directly employing agents would lead to limited bug reproduction performance, due to entangled subtasks, lengthy retrieved context, and unregulated actions. To mitigate the challenges, we propose an Automated gEneral buG reproductIon Scripts generation framework, named AEGIS, which is the first agent-based framework for the task. AEGIS mainly contains two modules: (1) A concise context construction module, which aims to guide the code agent in extracting structured information from issue descriptions, identifying issue-related code with detailed explanations, and integrating these elements to construct the concise context; (2) A FSM-based multi-feedback optimization module to further regulate the behavior of the code agent within the finite state machine (FSM), ensuring a controlled and efficient script generation process based on multi-dimensional feedback. Extensive experiments on the public benchmark dataset show that AEGIS outperforms the state-of-the-art baseline by 23.0% in F->P metric. In addition, the bug reproduction scripts generated by AEGIS can improve the relative resolved rate of Agentless by 12.5%.

Figures (7)

• Figure 1: Examples for illustrating the challenges of code agents in general bug reproduction. Turn X represents the relative order of different turns.
• Figure 2: The architecture of AEGIS. AEGIS mainly consists of two modules driven by the Reproducer Agent: a concise context construction module and a FSM-based multi-feedback optimization module. The buggy code and unit tests retrieved by the Searcher Agent, issue description, and the project codebase are the input of AEGIS. The output is the reproduction script.
• Figure 3: The example of the concise context, including the structured issue information, issue-related code, and the relevance explanation.
• Figure 4: The architecture of our designed finite state machine (FSM). (a), (b), and (c) illustrate the state transition diagram, definition of FSM, and transition function table of the designed FSM, respectively.
• Figure 5: (a) illustrates the impact of the number of restarts and edits per restart on AEGIS. (b) shows the successfully reproduced bugs by AEGIS against AgentBaseline.