PromptFlow: Training Prompts Like Neural Networks
Jingyi Wang, Hongyuan Zhu, Ye Niu, Yunhui Deng
TL;DR
PromptFlow addresses the labor and rigidity of manual and static prompt engineering by introducing a modular, TensorFlow-inspired framework that optimizes prompts at a fine-grained, meta-prompt level. It couples a rich operator library with gradient-based MSGD and reinforcement-learning–driven MSGD-RL optimizers to automatically discover efficient refinement trajectories and recycle prior experience. Across NER, CLS, and MRC tasks, PromptFlow consistently improves performance over strong baselines, with notable gains in NER and CLS and informative insights into operator-task preferences. The work enables scalable, data-efficient prompt optimization with practical impact for deploying LLMs in diverse domains, while highlighting future work on efficiency and broader task coverage.
Abstract
Large Language Models (LLMs) have demonstrated profound impact on Natural Language Processing (NLP) tasks. However, their effective deployment across diverse domains often require domain-specific adaptation strategies, as generic models may underperform when faced with specialized data distributions. Recent advances in prompt engineering (PE) offer a promising alternative to extensive retraining by refining input instructions to align LLM outputs with task objectives. This paradigm has emerged as a rapid and versatile approach for model fine-tuning. Despite its potential, manual prompt design remains labor-intensive and heavily depends on specialized expertise, often requiring iterative human effort to achieve optimal formulations. To address this limitation, automated prompt engineering methodologies have been developed to systematically generate task-specific prompts. However, current implementations predominantly employ static update rules and lack mechanisms for dynamic strategy selection, resulting in suboptimal adaptation to varying NLP task requirements. Furthermore, most methods treat and update the whole prompts at each step, without considering editing prompt sections at a finer granularity. At last, in particular, the problem of how to recycle experience in LLM is still underexplored. To this end, we propose the PromptFlow, a modular training framework inspired by TensorFlow, which integrates meta-prompts, operators, optimization, and evaluator. Our framework can be equipped with the latest optimization methods and autonomously explores optimal prompt refinement trajectories through gradient-based meta-learning, requiring minimal task-specific training data. Specifically, we devise a reinforcement learning method to recycle experience for LLM in the PE process. Finally, we conduct extensive experiments on various datasets, and demonstrate the effectiveness of PromptFlow.