DLPO: Towards a Robust, Efficient, and Generalizable Prompt Optimization Framework from a Deep-Learning Perspective

TL;DR

DLPO reframes prompt optimization as a gradient-based process guided by a forward and a backward engine, and introduces seven text-based strategies inspired by traditional deep learning to enhance robustness, efficiency, and generalization. The framework systematically reduces update instability with TextualLearningRate, TextualDropout, and TextualSimulatedAnnealing, accelerates convergence with TextualLearningRateDecay, TextualMomentum, and TextualContrastiveLearning, and controls prompt complexity with TextualRegularization. Across GSM8K, BigGSM, BBH, MATH, and MGSM, DLPO achieves state-of-the-art results, outperforming TextGrad, APO, and even human-crafted prompts on several benchmarks. The work provides practical guidance and a publicly available implementation for robust, efficient, and generalizable automated prompt optimization in real-world settings.

Abstract

Large Language Models (LLMs) have achieved remarkable success across diverse tasks, largely driven by well-designed prompts. However, crafting and selecting such prompts often requires considerable human effort, significantly limiting its scalability. To mitigate this, recent studies have explored automated prompt optimization as a promising solution. Despite these efforts, existing methods still face critical challenges in robustness, efficiency, and generalization. To systematically address these challenges, we first conduct an empirical analysis to identify the limitations of current reflection-based prompt optimization paradigm. Building on these insights, we propose 7 innovative approaches inspired by traditional deep learning paradigms for prompt optimization (DLPO), seamlessly integrating these concepts into text-based gradient optimization. Through these advancements, we progressively tackle the aforementioned challenges and validate our methods through extensive experimentation. We hope our study not only provides valuable guidance for future research but also offers a comprehensive understanding of the challenges and potential solutions in prompt optimization. Our code is available at https://github.com/sfasfaffa/DLPO.

Figures (6)

• Figure 1: Comparison between traditional reflection-based prompt optimization methods and DLPO, which incorporates 7 innovative approaches to progressively enhance the robustness, efficiency, and generalizability of prompt optimization.
• Figure 2: Current reflection-based paradigm for prompt optimization.
• Figure 3: a, b, c respectively show the validation-set accuracy of 3 different seeds and their mean values and standard deviations on BBH, GSM8K, and BigGSM environments. The mean values are represented by black solid lines, and the standard deviations are indicated by red shaded areas. d shows the Training-set and Validation-set mean accuracy results of 3 different seeds on GSM8K, along with their standard deviations. To make the image clearer, we use $\frac{1}{2}$ standard deviation as the shaded area.
• Figure 4: a, mean results of 3 different seeds for Tlr+Tdo and Naive on GSM8K. We use $\frac{1}{2}$ of the standard deviation as the shaded area. b, mean results of 3 different seeds for Tsa+Tlr+Tdo, Tlr+Tdo and Naive on BBH. To make the image clearer, we use $\frac{1}{4}$ of the standard deviation as the shaded area.
• Figure 5: a, b, mean results of 3 different seeds for Tsa+Tlrd and Tsa+Tlr on validation set of BigGSM and MGSM environment. We use $\frac{1}{2}$ of the standard deviation as the shaded area.