Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Plum: Prompt Learning using Metaheuristic

Rui Pan, Shuo Xing, Shizhe Diao, Wenhe Sun, Xiang Liu, Kashun Shum, Renjie Pi, Jipeng Zhang, Tong Zhang

TL;DR

The paper frames prompt learning for LLMs as a discrete, gradient-free black-box optimization problem and introduces Plum, a general metaheuristic-based framework. Plum defines a neighborhood, a chosen metaheuristic, and scoring functions to automatically search for effective prompts, enabling automatic, discrete, interpretable optimization. Through implementations of six algorithms (Hill Climbing, Simulated Annealing, Genetic Algorithm variants, Tabu Search, and Harmony Search), Plum demonstrates competitive performance in white-box and black-box settings and reveals human-understandable prompt patterns for reasoning and image generation. The work also explores prompt discovery, efficiency considerations, and potential integration with LLM-driven mutators, highlighting Plum as a flexible, extensible template for prompt optimization across modalities and tasks.

Abstract

Since the emergence of large language models, prompt learning has become a popular method for optimizing and customizing these models. Special prompts, such as Chain-of-Thought, have even revealed previously unknown reasoning capabilities within these models. However, the progress of discovering effective prompts has been slow, driving a desire for general prompt optimization methods. Unfortunately, few existing prompt learning methods satisfy the criteria of being truly "general", i.e., automatic, discrete, black-box, gradient-free, and interpretable all at once. In this paper, we introduce metaheuristics, a branch of discrete non-convex optimization methods with over 100 options, as a promising approach to prompt learning. Within our paradigm, we test six typical methods: hill climbing, simulated annealing, genetic algorithms with/without crossover, tabu search, and harmony search, demonstrating their effectiveness in white-box and black-box prompt learning. Furthermore, we show that these methods can be used to discover more human-understandable prompts that were previously unknown in both reasoning and image generation tasks, opening the door to a cornucopia of possibilities in prompt optimization. We release all the codes in \url{https://github.com/research4pan/Plum}.

Plum: Prompt Learning using Metaheuristic

TL;DR

The paper frames prompt learning for LLMs as a discrete, gradient-free black-box optimization problem and introduces Plum, a general metaheuristic-based framework. Plum defines a neighborhood, a chosen metaheuristic, and scoring functions to automatically search for effective prompts, enabling automatic, discrete, interpretable optimization. Through implementations of six algorithms (Hill Climbing, Simulated Annealing, Genetic Algorithm variants, Tabu Search, and Harmony Search), Plum demonstrates competitive performance in white-box and black-box settings and reveals human-understandable prompt patterns for reasoning and image generation. The work also explores prompt discovery, efficiency considerations, and potential integration with LLM-driven mutators, highlighting Plum as a flexible, extensible template for prompt optimization across modalities and tasks.

Abstract

Since the emergence of large language models, prompt learning has become a popular method for optimizing and customizing these models. Special prompts, such as Chain-of-Thought, have even revealed previously unknown reasoning capabilities within these models. However, the progress of discovering effective prompts has been slow, driving a desire for general prompt optimization methods. Unfortunately, few existing prompt learning methods satisfy the criteria of being truly "general", i.e., automatic, discrete, black-box, gradient-free, and interpretable all at once. In this paper, we introduce metaheuristics, a branch of discrete non-convex optimization methods with over 100 options, as a promising approach to prompt learning. Within our paradigm, we test six typical methods: hill climbing, simulated annealing, genetic algorithms with/without crossover, tabu search, and harmony search, demonstrating their effectiveness in white-box and black-box prompt learning. Furthermore, we show that these methods can be used to discover more human-understandable prompts that were previously unknown in both reasoning and image generation tasks, opening the door to a cornucopia of possibilities in prompt optimization. We release all the codes in \url{https://github.com/research4pan/Plum}.
Paper Structure (52 sections, 1 equation, 6 figures, 8 tables, 1 algorithm)

This paper contains 52 sections, 1 equation, 6 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Prompt Learning
  4. Black-Box Prompt Learning
  5. Metaheuristics
  6. Methods
  7. Plum in General
  8. Plum in Practice
  9. Plum-GA-C
  10. Stage 1: Parsing
  11. Stage 2: Editing
  12. Mutation
  13. Crossover
  14. Stage 3: Scoring
  15. Plum-HS
  16. ...and 37 more sections

Figures (6)

  • Figure 1: Illustration of Plum-GA-C and its possible extension with LLMs.
  • Figure 2: Performance of Plum-HS across different iterations on subtasks of Natural-Instructions dataset.
  • Figure 3: Illustration of the improved add operation and revision by language models
  • Figure 4: Illustration of the initial prompt for GSM8K with GPT3 (left) and GPT3.5 (right) as backbone model
  • Figure 5: Illustration of the improved add operation and revision with GPT3.5 as backbone model
  • ...and 1 more figures