Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Adaptive Software Agents for Debugging

Yacine Majdoub, Eya Ben Charrada, Haifa Touati

TL;DR

The paper tackles inefficiencies in static, multi-agent debugging with LLMs by proposing an adaptive agent design where a central main agent analyzes buggy code, plans the debugging strategy, profiles specialized agents, and dynamically instantiates them based on task complexity. Specialized agents collaborate under the main agent to execute the plan, with iterative validation guiding re-plans to avoid repeats. Preliminary evaluation on 50 Python buggy instances across four LLMs shows the adaptive approach improves bug-fix rates by 6–18% and scales resource usage to task difficulty (1 agent for simple tasks, up to 5 for complex ones). This work demonstrates the feasibility and benefits of adaptive, autonomous planning for software debugging and outlines directions for broader application and autonomous goal-driven behavior in software agents.

Abstract

Using multiple agents was found to improve the debugging capabilities of Large Language Models. However, increasing the number of LLM-agents has several drawbacks such as increasing the running costs and rising the risk for the agents to lose focus. In this work, we propose an adaptive agentic design, where the number of agents and their roles are determined dynamically based on the characteristics of the task to be achieved. In this design, the agents roles are not predefined, but are generated after analyzing the problem to be solved. Our initial evaluation shows that, with the adaptive design, the number of agents that are generated depends on the complexity of the buggy code. In fact, for simple code with mere syntax issues, the problem was usually fixed using one agent only. However, for more complex problems, we noticed the creation of a higher number of agents. Regarding the effectiveness of the fix, we noticed an average improvement of 11% compared to the one-shot prompting. Given these promising results, we outline future research directions to improve our design for adaptive software agents that can autonomously plan and conduct their software goals.

Towards Adaptive Software Agents for Debugging

TL;DR

The paper tackles inefficiencies in static, multi-agent debugging with LLMs by proposing an adaptive agent design where a central main agent analyzes buggy code, plans the debugging strategy, profiles specialized agents, and dynamically instantiates them based on task complexity. Specialized agents collaborate under the main agent to execute the plan, with iterative validation guiding re-plans to avoid repeats. Preliminary evaluation on 50 Python buggy instances across four LLMs shows the adaptive approach improves bug-fix rates by 6–18% and scales resource usage to task difficulty (1 agent for simple tasks, up to 5 for complex ones). This work demonstrates the feasibility and benefits of adaptive, autonomous planning for software debugging and outlines directions for broader application and autonomous goal-driven behavior in software agents.

Abstract

Using multiple agents was found to improve the debugging capabilities of Large Language Models. However, increasing the number of LLM-agents has several drawbacks such as increasing the running costs and rising the risk for the agents to lose focus. In this work, we propose an adaptive agentic design, where the number of agents and their roles are determined dynamically based on the characteristics of the task to be achieved. In this design, the agents roles are not predefined, but are generated after analyzing the problem to be solved. Our initial evaluation shows that, with the adaptive design, the number of agents that are generated depends on the complexity of the buggy code. In fact, for simple code with mere syntax issues, the problem was usually fixed using one agent only. However, for more complex problems, we noticed the creation of a higher number of agents. Regarding the effectiveness of the fix, we noticed an average improvement of 11% compared to the one-shot prompting. Given these promising results, we outline future research directions to improve our design for adaptive software agents that can autonomously plan and conduct their software goals.

Paper Structure

This paper contains 31 sections, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. background and motivation
  3. Reasoning with LLMs
  4. Chain-of-Thought
  5. Self-consistency
  6. Iterative-reflection
  7. Planning
  8. Tool use
  9. Multi-agent collaboration
  10. Use of LLM reasoning for debugging
  11. Limitations of debugging with LLMs
  12. Meaning of agenticness
  13. Agenticness of current solutions
  14. Need for adaptive designs
  15. Designing Adaptive agents
  16. ...and 16 more sections

Figures (3)

  • Figure 1: Overview of the approach
  • Figure 2: Number of created specialized agents using Llama3 and GPT-4 for different complexity levels
  • Figure 3: Number of needed iterations using Llama3 and GPT-4 for different complexity levels