Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Goal-oriented Prompt Engineering for Large Language Models: A Survey

Haochen Li, Jonathan Leung, Zhiqi Shen

TL;DR

From a review of 50 representative studies, it is demonstrated that a goal-oriented prompt formulation, which guides LLMs to follow established human logical thinking, significantly improves the performance of LLMs.

Abstract

Large Language Models (LLMs) have shown prominent performance in various downstream tasks and prompt engineering plays a pivotal role in optimizing LLMs' performance. This paper, not only as an overview of current prompt engineering methods, but also aims to highlight the limitation of designing prompts based on an anthropomorphic assumption that expects LLMs to think like humans. From our review of 50 representative studies, we demonstrate that a goal-oriented prompt formulation, which guides LLMs to follow established human logical thinking, significantly improves the performance of LLMs. Furthermore, We introduce a novel taxonomy that categorizes goal-oriented prompting methods into five interconnected stages and we demonstrate the broad applicability of our framework. With four future directions proposed, we hope to further emphasize the power and potential of goal-oriented prompt engineering in all fields.

Towards Goal-oriented Prompt Engineering for Large Language Models: A Survey

TL;DR

From a review of 50 representative studies, it is demonstrated that a goal-oriented prompt formulation, which guides LLMs to follow established human logical thinking, significantly improves the performance of LLMs.

Abstract

Large Language Models (LLMs) have shown prominent performance in various downstream tasks and prompt engineering plays a pivotal role in optimizing LLMs' performance. This paper, not only as an overview of current prompt engineering methods, but also aims to highlight the limitation of designing prompts based on an anthropomorphic assumption that expects LLMs to think like humans. From our review of 50 representative studies, we demonstrate that a goal-oriented prompt formulation, which guides LLMs to follow established human logical thinking, significantly improves the performance of LLMs. Furthermore, We introduce a novel taxonomy that categorizes goal-oriented prompting methods into five interconnected stages and we demonstrate the broad applicability of our framework. With four future directions proposed, we hope to further emphasize the power and potential of goal-oriented prompt engineering in all fields.
Paper Structure (29 sections, 2 equations, 2 figures, 10 tables)

This paper contains 29 sections, 2 equations, 2 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Preliminary
  3. Taxonomy of Methods
  4. Decomposing Goal into Sub-goal Sequences
  5. Iterative decomposition.
  6. Plan-then-execute decomposition.
  7. External decomposition.
  8. Action Selection for Attaining Sub-goals
  9. Constrain-then-select.
  10. Select-then-mapping.
  11. Executing Actions to Get Sub-goal Results
  12. Evaluating Sub-goal Results
  13. Feedback from LLMs.
  14. Feedback from external tools.
  15. Further Selection of Valuable Sub-goals
  16. ...and 14 more sections

Figures (2)

  • Figure 1: An overview of the goal-oriented framework for prompting LLMs taking solving a math word problem as an example. (1) Decomposing goal into sub-goal sequences. (2) Action selection for attaining sub-goals. (3) Executing actions to get sub-goal results. (4) Evaluating sub-goal results. (5) Further selection of valuable sub-goals. Note that stages (2)(3)(4) are taken for all the decomposed sub-goals.
  • Figure 2: Illustration of approaches for valuable sub-goal selection. (a) CoT selects all sub-goals in one sub-goal sequence. (b) Self-consistency, a variant of CoT, selects sub-goals based on majority votes. (c) MCR, a variant of CoT, selects sub-goals from multiple sub-goal sequences. (d) Recmind and Selection-Inference, variants of CoT, select one sub-goal from candidates at each step. (e) ToT explores sub-goals from a tree structure space. (f) GDP-Zero and RAP, variants of ToT, introduce backpropagation to ToT to balance exploration and exploitation. (g) GoT models all sub-goals in a graph structure space.