Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ROAD: Reflective Optimization via Automated Debugging for Zero-Shot Agent Alignment

Natchaya Temyingyong, Daman Jain, Neeraj Kumarsahu, Prabhat Kumar, Rachata Phondi, Wachiravit Modecrua, Krittanon Kaewtawee, Krittin Pachtrachai, Touchapon Kraisingkorn

TL;DR

ROAD addresses the data-scarce reality of deploying LLM-based agents by reframing optimization as a dynamic debugging loop driven by production failures. It employs a multi-agent system (Analyzer, Optimizer, Coach) to extract root causes, synthesize a deterministic Decision Tree Protocol, and iteratively evolve the primary agent's prompt. Across both a standardized academic benchmark and a live KM engine, ROAD demonstrates data-efficient gains, including a 5.6% increase in success rate and a 3.8% rise in search accuracy within three automated iterations, plus about 19% improvement on complex retail tasks. This work highlights that mimicking human failure analysis and patching can achieve reliable, scalable improvements without costly RL training, enabling rapid deployment in data-scarce environments.

Abstract

Automatic Prompt Optimization (APO) has emerged as a critical technique for enhancing Large Language Model (LLM) performance, yet current state-of-the-art methods typically rely on large, labeled gold-standard development sets to compute fitness scores for evolutionary or Reinforcement Learning (RL) approaches. In real-world software engineering, however, such curated datasets are rarely available during the initial cold start of agent development, where engineers instead face messy production logs and evolving failure modes. We present ROAD (Reflective Optimization via Automated Debugging), a novel framework that bypasses the need for refined datasets by treating optimization as a dynamic debugging investigation rather than a stochastic search. Unlike traditional mutation strategies, ROAD utilizes a specialized multi-agent architecture, comprising an Analyzer for root-cause analysis, an Optimizer for pattern aggregation, and a Coach for strategy integration, to convert unstructured failure logs into robust, structured Decision Tree Protocols. We evaluated ROAD across both a standardized academic benchmark and a live production Knowledge Management engine. Experimental results demonstrate that ROAD is highly sample-efficient, achieving a 5.6 percent increase in success rate (73.6 percent to 79.2 percent) and a 3.8 percent increase in search accuracy within just three automated iterations. Furthermore, on complex reasoning tasks in the retail domain, ROAD improved agent performance by approximately 19 percent relative to the baseline. These findings suggest that mimicking the human engineering loop of failure analysis and patching offers a viable, data-efficient alternative to resource-intensive RL training for deploying reliable LLM agents.

ROAD: Reflective Optimization via Automated Debugging for Zero-Shot Agent Alignment

TL;DR

ROAD addresses the data-scarce reality of deploying LLM-based agents by reframing optimization as a dynamic debugging loop driven by production failures. It employs a multi-agent system (Analyzer, Optimizer, Coach) to extract root causes, synthesize a deterministic Decision Tree Protocol, and iteratively evolve the primary agent's prompt. Across both a standardized academic benchmark and a live KM engine, ROAD demonstrates data-efficient gains, including a 5.6% increase in success rate and a 3.8% rise in search accuracy within three automated iterations, plus about 19% improvement on complex retail tasks. This work highlights that mimicking human failure analysis and patching can achieve reliable, scalable improvements without costly RL training, enabling rapid deployment in data-scarce environments.

Abstract

Automatic Prompt Optimization (APO) has emerged as a critical technique for enhancing Large Language Model (LLM) performance, yet current state-of-the-art methods typically rely on large, labeled gold-standard development sets to compute fitness scores for evolutionary or Reinforcement Learning (RL) approaches. In real-world software engineering, however, such curated datasets are rarely available during the initial cold start of agent development, where engineers instead face messy production logs and evolving failure modes. We present ROAD (Reflective Optimization via Automated Debugging), a novel framework that bypasses the need for refined datasets by treating optimization as a dynamic debugging investigation rather than a stochastic search. Unlike traditional mutation strategies, ROAD utilizes a specialized multi-agent architecture, comprising an Analyzer for root-cause analysis, an Optimizer for pattern aggregation, and a Coach for strategy integration, to convert unstructured failure logs into robust, structured Decision Tree Protocols. We evaluated ROAD across both a standardized academic benchmark and a live production Knowledge Management engine. Experimental results demonstrate that ROAD is highly sample-efficient, achieving a 5.6 percent increase in success rate (73.6 percent to 79.2 percent) and a 3.8 percent increase in search accuracy within just three automated iterations. Furthermore, on complex reasoning tasks in the retail domain, ROAD improved agent performance by approximately 19 percent relative to the baseline. These findings suggest that mimicking the human engineering loop of failure analysis and patching offers a viable, data-efficient alternative to resource-intensive RL training for deploying reliable LLM agents.
Paper Structure (50 sections, 3 figures, 2 tables, 1 algorithm)

This paper contains 50 sections, 3 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. System Architecture
  5. The Optimization Pipeline
  6. Phase 1: Execution and Selective Filtering
  7. Phase 2: Semantic Root-Cause Analysis
  8. Phase 3: Pattern Recognition and Decision Tree Synthesis
  9. Phase 4: Targeted Prompt Evolution
  10. Algorithmic Formalization
  11. Experiments
  12. Experimental Setup 1: Academic Benchmarking ($\tau^2$-bench)
  13. Benchmark Selection
  14. Domain Specifics (Retail)
  15. Domain Specifics (Retail)
  16. ...and 35 more sections

Figures (3)

  • Figure 1: The ROAD Framework Pipeline. (1) The Contestant executes tasks; (2) Failures are filtered; (3) The Analyzer diagnoses root causes; (4) The Optimizer synthesizes a Decision Tree ($\mathcal{T}$); (5) The Coach updates the system prompt ($P$).
  • Figure 2: Trajectory of performance improvements over ROAD iterations. While o4-mini peaked early at Iteration 2, Qwen3-4B exhibited non-monotonic learning behavior before converging to a higher maxima at Iteration 6.
  • Figure 3: Performance comparison on real-world knowledge management cases. The ROAD framework (Iteration 3) outperformed the baseline in both overall task success and document retrieval accuracy.