How to Auto-optimize Prompts for Domain Tasks? Adaptive Prompting and Reasoning through Evolutionary Domain Knowledge Adaptation

TL;DR

The paper tackles the challenge of adapting large language models to domain-specific tasks without fine-tuning by introducing EGO-Prompt, an Evolutionary Graph Optimization framework that integrates expert-constructed Semantic Causal Graphs with prompt engineering. It decomposes domain reasoning into instance-specific guidance $z^{\!*}(x,\mathcal{G})$ generated from the SCG and final reasoning conditioned on this guidance, with iterative textual-gradient updates to both the graph and prompts. Empirical results across public health, transportation, and human-behavior tasks show consistent F1 gains (7.32%–12.61%), enable smaller models to match larger ones at substantially lower cost, and yield refined SCGs that enhance interpretability. The work advances domain-adaptive prompting by enabling automatic SCG correction and dynamic graph-based reasoning, with potential extensions to dynamic RAG, knowledge-graph construction, and causal discovery for real-world decision support.

Abstract

Designing optimal prompts and reasoning processes for large language models (LLMs) on domain-specific tasks is both necessary and challenging in real-world applications. Determining how to integrate domain knowledge, enhance reasoning efficiency, and even provide domain experts with refined knowledge integration hints are particularly crucial yet unresolved tasks. In this research, we propose Evolutionary Graph Optimization for Prompting (EGO-Prompt), an automated framework to designing better prompts, efficient reasoning processes and providing enhanced causal-informed process. EGO-Prompt begins with a general prompt and fault-tolerant initial Semantic Causal Graph (SCG) descriptions, constructed by human experts, which is then automatically refined and optimized to guide LLM reasoning. Recognizing that expert-defined SCGs may be partial or imperfect and that their optimal integration varies across LLMs, EGO-Prompt integrates a novel causal-guided textual gradient process in two steps: first, generating nearly deterministic reasoning guidance from the SCG for each instance, and second, adapting the LLM to effectively utilize the guidance alongside the original input. The iterative optimization algorithm further refines both the SCG and the reasoning mechanism using textual gradients with ground-truth. We tested the framework on real-world public health, transportation and human behavior tasks. EGO-Prompt achieves 7.32%-12.61% higher F1 than cutting-edge methods, and allows small models to reach the performence of larger models at under 20% of the original cost. It also outputs a refined, domain-specific SCG that improves interpretability.

Figures (8)

• Figure 1: Overview of the proposed EGO-Prompt. (a) LLMs often struggle with domain-specific tasks due to the optimal prompt design and domain knowledge gap. Existing methods rely on the external database or established graph. In comparison, EGO-Prompt evolutionarily incorporates expert knowledge with minimal cost. (b) We represent external knowledge as a graph-based structure. A graph-enhanced prompt is then generated to guide the LLM’s reasoning. Both the graph and the prompt are iteratively optimized using textual gradients from ground-truth data.
• Figure 2: The optimization process of EGO-Prompt. The graph model generates reasoning guidance from the SCG and input prompt, which the prediction model uses to produce an output. Textual gradients are used to update the system prompt and refine the SCG through targeted operations.
• Figure 3: Mean F1 Score Across 3 Datasets.
• Figure 4: Mean F1 Score and Mean Price with Reasoning Models.
• Figure 5: Automatic SCG Correction for the Pandemic Dataset. Green line denotes newly added relations; red dash denotes deleted relation.