Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Survey on Prompting Techniques in LLMs

Prabin Bhandari

TL;DR

This survey examines prompting techniques for autoregressive large language models, outlining a taxonomy based on human effort (Hand-Crafted vs Automated) and the intended purpose (Task-Based, Generate-Auxiliary, Resource/Tools Augmented). It covers both classic handcrafted prompts and modern automated approaches, including discrete hard prompts and continuous soft prompts such as Prefix Tuning and P-tuning, as well as generation-based strategies like Chain-of-Thought, generated knowledge, and tool-augmented methods like ReAct and PAL. Key contributions include a structured literature review, discussion of emergent prompting phenomena (e.g., CoT, self-consistency, ToT), and a synthesis of open problems including robustness to sub-optimal prompts, costs of large-scalePrompt tuning, and challenges with structured data and closed-source models. The paper highlights the practical significance of prompting as a scalable alternative to fine-tuning, while calling for continued research on robustness, efficiency, and the integration of external tools and structured inputs to broaden applicability in real-world NLP tasks.

Abstract

Autoregressive Large Language Models have transformed the landscape of Natural Language Processing. Pre-train and prompt paradigm has replaced the conventional approach of pre-training and fine-tuning for many downstream NLP tasks. This shift has been possible largely due to LLMs and innovative prompting techniques. LLMs have shown great promise for a variety of downstream tasks owing to their vast parameters and huge datasets that they are pre-trained on. However, in order to fully realize their potential, their outputs must be guided towards the desired outcomes. Prompting, in which a specific input or instruction is provided to guide the LLMs toward the intended output, has become a tool for achieving this goal. In this paper, we discuss the various prompting techniques that have been applied to fully harness the power of LLMs. We present a taxonomy of existing literature on prompting techniques and provide a concise survey based on this taxonomy. Further, we identify some open problems in the realm of prompting in autoregressive LLMs which could serve as a direction for future research.

A Survey on Prompting Techniques in LLMs

TL;DR

This survey examines prompting techniques for autoregressive large language models, outlining a taxonomy based on human effort (Hand-Crafted vs Automated) and the intended purpose (Task-Based, Generate-Auxiliary, Resource/Tools Augmented). It covers both classic handcrafted prompts and modern automated approaches, including discrete hard prompts and continuous soft prompts such as Prefix Tuning and P-tuning, as well as generation-based strategies like Chain-of-Thought, generated knowledge, and tool-augmented methods like ReAct and PAL. Key contributions include a structured literature review, discussion of emergent prompting phenomena (e.g., CoT, self-consistency, ToT), and a synthesis of open problems including robustness to sub-optimal prompts, costs of large-scalePrompt tuning, and challenges with structured data and closed-source models. The paper highlights the practical significance of prompting as a scalable alternative to fine-tuning, while calling for continued research on robustness, efficiency, and the integration of external tools and structured inputs to broaden applicability in real-world NLP tasks.

Abstract

Autoregressive Large Language Models have transformed the landscape of Natural Language Processing. Pre-train and prompt paradigm has replaced the conventional approach of pre-training and fine-tuning for many downstream NLP tasks. This shift has been possible largely due to LLMs and innovative prompting techniques. LLMs have shown great promise for a variety of downstream tasks owing to their vast parameters and huge datasets that they are pre-trained on. However, in order to fully realize their potential, their outputs must be guided towards the desired outcomes. Prompting, in which a specific input or instruction is provided to guide the LLMs toward the intended output, has become a tool for achieving this goal. In this paper, we discuss the various prompting techniques that have been applied to fully harness the power of LLMs. We present a taxonomy of existing literature on prompting techniques and provide a concise survey based on this taxonomy. Further, we identify some open problems in the realm of prompting in autoregressive LLMs which could serve as a direction for future research.
Paper Structure (26 sections, 2 equations, 4 figures)

This paper contains 26 sections, 2 equations, 4 figures.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Language Models
  4. Statistical language models (SLM)
  5. Neural language models (NLM)
  6. Pre-trained language models (PLM)
  7. Left-to-Right PLM
  8. Masked PLM
  9. Encoder-Decoder PLM
  10. Large language models (LLM)
  11. Prompting
  12. Area Taxonomy
  13. Taxonomy-Based Survey
  14. Human Effort
  15. Hand-crafted Prompts
  16. ...and 11 more sections

Figures (4)

  • Figure 1: Architecture of different PLMs. Image from wang2022pre.
  • Figure 2: Area Taxonomy of prompting methods in autoregressive LLMs.
  • Figure 3: Chain of thought prompting. Image from wei2022chain.
  • Figure 4: Self-consistency methodology for CoT prompting in comparison to CoT prompting with greedy decoding. Image from yao2023tree.