DelvePO: Direction-Guided Self-Evolving Framework for Flexible Prompt Optimization

TL;DR

DelvePO introduces a direction-guided, self-evolving framework for flexible prompt optimization that decouples prompts into functional components and employs working memory to stabilize and steer evolution across tasks and LLMs. By integrating Component Memory and Prompt Memory, it guides task and solution evolutions to explore effective component values while mitigating premature convergence. Extensive experiments across 11 datasets and multiple LLMs show DelvePO consistently outperforms manual prompts and existing PO methods, with notable gains on both open- and closed-source models. The approach offers enhanced interpretability and transferability, paving the way for efficient, scalable prompt optimization in diverse domains.

Abstract

Prompt Optimization has emerged as a crucial approach due to its capabilities in steering Large Language Models to solve various tasks. However, current works mainly rely on the random rewriting ability of LLMs, and the optimization process generally focus on specific influencing factors, which makes it easy to fall into local optimum. Besides, the performance of the optimized prompt is often unstable, which limits its transferability in different tasks. To address the above challenges, we propose $\textbf{DelvePO}$ ($\textbf{D}$irection-Guid$\textbf{e}$d Se$\textbf{l}$f-E$\textbf{v}$olving Framework for Fl$\textbf{e}$xible $\textbf{P}$rompt $\textbf{O}$ptimization), a task-agnostic framework to optimize prompts in self-evolve manner. In our framework, we decouple prompts into different components that can be used to explore the impact that different factors may have on various tasks. On this basis, we introduce working memory, through which LLMs can alleviate the deficiencies caused by their own uncertainties and further obtain key insights to guide the generation of new prompts. Extensive experiments conducted on different tasks covering various domains for both open- and closed-source LLMs, including DeepSeek-R1-Distill-Llama-8B, Qwen2.5-7B-Instruct and GPT-4o-mini. Experimental results show that DelvePO consistently outperforms previous SOTA methods under identical experimental settings, demonstrating its effectiveness and transferability across different tasks.

Figures (25)

• Figure 1: The Framework of DelvePO. Initialization begins with predefined components, which are concatenated to form individual $p$; multiple individuals constitute the initial population Populations (old). At each step, one individual (Mutation only) or two individuals (Mutation and Crossover) are sampled, and the Sub-task determines the evolutionary direction (i.e., the mutated component type). Guided by Task-, Solution-, and Memory-Evolution modules, selected prompts are iteratively evolved, contrasting with unguided optimization. The new population Populations (new) is accumulated across epochs, and once the threshold is reached, the population is updated to initiate the next round of self-evolution.
• Figure 2: The mechanism of Task-Evolution and Solution-Evolution. Using the pseudo-prompt to explain the details of Task- and Solution-Evolution.
• Figure 3: Average time-consuming (GPU hours) for one epoch of optimization on DeepSeek-R1-Distill-Llama-8B.
• Figure 4: Task-agnostic template for generating component values corresponding to the given component types. The following part of the figure is the prompt to generate content for Component "role" using the casual judgement task as an example.
• Figure 5: Template for initializing prompt populations. It is also used in the construction of Prompts Memory, that is, injecting discrete components into the template to obtain a continuous form prompt. The above shows the general form, while the two below provide illustrative examples.