Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Sequential Optimal Learning Approach to Automated Prompt Engineering in Large Language Models

Shuyang Wang, Somayeh Moazeni, Diego Klabjan

TL;DR

This paper proposes an optimal learning framework for automated prompt engineering, designed to sequentially identify effective prompt features while efficiently allocating a limited evaluation budget, and introduces a feature-based method to express prompts, which significantly broadens the search space.

Abstract

Designing effective prompts is essential to guiding large language models (LLMs) toward desired responses. Automated prompt engineering aims to reduce reliance on manual effort by streamlining the design, refinement, and optimization of natural language prompts. This paper proposes an optimal learning framework for automated prompt engineering, designed to sequentially identify effective prompt features while efficiently allocating a limited evaluation budget. We introduce a feature-based method to express prompts, which significantly broadens the search space. Bayesian regression is employed to utilize correlations among similar prompts, accelerating the learning process. To efficiently explore the large space of prompt features for a high quality prompt, we adopt the forward-looking Knowledge-Gradient (KG) policy for sequential optimal learning. The KG policy is computed efficiently by solving mixed-integer second-order cone optimization problems, making it scalable and capable of accommodating prompts characterized only through constraints. We demonstrate that our method significantly outperforms a set of benchmark strategies assessed on instruction induction tasks. The results highlight the advantages of using the KG policy for prompt learning given a limited evaluation budget. Our framework provides a solution to deploying automated prompt engineering in a wider range applications where prompt evaluation is costly.

A Sequential Optimal Learning Approach to Automated Prompt Engineering in Large Language Models

TL;DR

This paper proposes an optimal learning framework for automated prompt engineering, designed to sequentially identify effective prompt features while efficiently allocating a limited evaluation budget, and introduces a feature-based method to express prompts, which significantly broadens the search space.

Abstract

Designing effective prompts is essential to guiding large language models (LLMs) toward desired responses. Automated prompt engineering aims to reduce reliance on manual effort by streamlining the design, refinement, and optimization of natural language prompts. This paper proposes an optimal learning framework for automated prompt engineering, designed to sequentially identify effective prompt features while efficiently allocating a limited evaluation budget. We introduce a feature-based method to express prompts, which significantly broadens the search space. Bayesian regression is employed to utilize correlations among similar prompts, accelerating the learning process. To efficiently explore the large space of prompt features for a high quality prompt, we adopt the forward-looking Knowledge-Gradient (KG) policy for sequential optimal learning. The KG policy is computed efficiently by solving mixed-integer second-order cone optimization problems, making it scalable and capable of accommodating prompts characterized only through constraints. We demonstrate that our method significantly outperforms a set of benchmark strategies assessed on instruction induction tasks. The results highlight the advantages of using the KG policy for prompt learning given a limited evaluation budget. Our framework provides a solution to deploying automated prompt engineering in a wider range applications where prompt evaluation is costly.
Paper Structure (22 sections, 12 equations, 7 figures, 4 tables, 2 algorithms)

This paper contains 22 sections, 12 equations, 7 figures, 4 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Generation-then-selection Methods
  4. Edit-based Methods
  5. Prompt Learning Methods
  6. Gradient-based Methods
  7. Sequential Optimal Prompt Learning
  8. Feature-Based Prompt Representation
  9. Prompt Evaluation
  10. Knowledge State Update
  11. Prompt Representation Selection Policy
  12. KG Prompt Selection Policy
  13. Computational Experiments
  14. Feature-Based Prompt Representation.
  15. Prompt Evaluation.
  16. ...and 7 more sections

Figures (7)

  • Figure 1: SOPL: Sequential optimal prompt learning for automated prompt engineering
  • Figure 2: Meta Prompt Templates
  • Figure 3: Paraphrasing Template
  • Figure 4: Evaluation Template
  • Figure 5: Test performance on 13 tasks for different methods. The height of each bar represents the average test score and the error bar represents the standard deviation across 20 replications with different random seeds
  • ...and 2 more figures