MARS: Multi-Agent Adaptive Reasoning with Socratic Guidance for Automated Prompt Optimization

TL;DR

MARS introduces a novel multi-agent framework for Automated Prompt Optimization that frames the task as a POMDP and uses a Planner plus a Socratic Teacher-Critic-Student loop to iteratively refine prompts. By integrating task planning, interactive reasoning, and downstream feedback, MARS achieves consistent improvements in performance and efficiency across diverse general and domain-specific benchmarks. The approach offers strong interpretability through traceable planning trajectories and dialogue-based refinements, supported by theoretical guarantees on improvement and stability. Empirical results demonstrate superior optimization quality, faster convergence, and robust generalization across base and target LLMs, highlighting its practical impact for scalable, adaptable prompt design.

Abstract

Large language models (LLMs) typically operate in a question-answering paradigm, where the quality of the input prompt critically affects the response. Automated Prompt Optimization (APO) aims to overcome the cognitive biases of manually crafted prompts and explore a broader prompt design space. However, existing APO methods often suffer from rigid template structures and inefficient exploration in the prompt space. To this end, we propose a Multi-Agent Adaptive Reasoning with Socratic guidance framework (MARS) for APO. MARS consists of five complementary agents and formulates the optimization process as a Partially Observable Markov Decision Process (POMDP), enabling adaptive prompt refinement through explicit state modeling and interactive feedback. Specifically, a Planner agent generates flexible optimization trajectories, a Teacher-Critic-Student triad engages in Socratic-style dialogue to iteratively optimize the prompt based on pseudo-gradient signals in the text space, and a Target agent executes the prompt in downstream tasks to provide performance feedback. MARS integrates reasoning, feedback, and state transition into a unified hidden-state evolution process, improving both the effectiveness and interpretability of optimization. Extensive experiments on multiple datasets demonstrate that MARS outperforms existing APO methods in terms of optimization performance, search efficiency, and interpretability.

Figures (7)

• Figure 1: Three different prompts along with their corresponding responses for the word sorting task.
• Figure 2: Comparison of APO strategies. Generation-search and meta-prompt. Multi-Agent Adaptive Reasoning enables dynamic, collaborative reasoning. Right: With GPT-4o, MARS outperforms all baselines on three benchmarks.
• Figure 3: The overall architecture of the MARS model. It consists of five LLM agents. The Planner agent that autonomously generates task-specific optimization trajectories, and a Teacher-Critic-Student Socratic dialogue mechanism that iteratively refines prompts, with the evaluation and iterative refinement process guided by feedback from the Target agent.
• Figure 4: A specific illustration of a Teacher-Critic-Student Socratic guidance dialogue pattern. The case shows the fifth step optimization iteration.
• Figure 5: Inference-time scaling law. The horizontal axis denotes the inference-time computational cost, while the vertical axis represents the average performances on all tasks.