A Systematic Survey of Automatic Prompt Optimization Techniques
Kiran Ramnath, Kang Zhou, Sheng Guan, Soumya Smruti Mishra, Xuan Qi, Zhengyuan Shen, Shuai Wang, Sangmin Woo, Sullam Jeoung, Yawei Wang, Haozhu Wang, Han Ding, Yuzhe Lu, Zhichao Xu, Yun Zhou, Balasubramaniam Srinivasan, Qiaojing Yan, Yueyan Chen, Haibo Ding, Panpan Xu, Lin Lee Cheong
TL;DR
Automatic Prompt Optimization (APO) tackles the challenge of eliciting reliable LLM behavior without accessing model parameters. The paper formalizes APO, introduces a five-part taxonomy, and provides a fine-grained framework to categorize seed strategies, evaluation feedback, candidate-generation techniques, filtering, and search depth. It surveys a wide spectrum of methods—from heuristic edits and LLM-driven feedback to meta-prompts and program-synthesis pipelines—while discussing theoretical bounds and practical challenges. The work aims to unify the field, guiding future research on task-agnostic APO, multimodal prompts, and agent-based systems, with implications for improving end-user experiences and robustness of prompt-based NLP systems.
Abstract
Since the advent of large language models (LLMs), prompt engineering has been a crucial step for eliciting desired responses for various Natural Language Processing (NLP) tasks. However, prompt engineering remains an impediment for end users due to rapid advances in models, tasks, and associated best practices. To mitigate this, Automatic Prompt Optimization (APO) techniques have recently emerged that use various automated techniques to help improve the performance of LLMs on various tasks. In this paper, we present a comprehensive survey summarizing the current progress and remaining challenges in this field. We provide a formal definition of APO, a 5-part unifying framework, and then proceed to rigorously categorize all relevant works based on their salient features therein. We hope to spur further research guided by our framework.