Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Survey of Prompt Engineering Methods in Large Language Models for Different NLP Tasks

Shubham Vatsal, Harsh Dubey

TL;DR

This survey addresses the problem of systematically understanding how prompting strategies enable LLMs to perform a wide range of NLP tasks without fine-tuning. It organizes 39 prompting methods into a comprehensive taxonomy and maps them to 29 NLP tasks, drawing on 44 papers to extract overall trends, insights, and potential SoTA approaches for each dataset. The key contributions include a detailed taxonomy diagram, dataset-level performance discussions, and a standardized view of prompt techniques across tasks, highlighting the growing importance of structured prompts, reasoning traces, and retrieval methods. The work's practical impact lies in guiding researchers and practitioners to select effective prompting strategies for specific tasks, datasets, and LLMs, while identifying gaps and directions for future research in prompt engineering.

Abstract

Large language models (LLMs) have shown remarkable performance on many different Natural Language Processing (NLP) tasks. Prompt engineering plays a key role in adding more to the already existing abilities of LLMs to achieve significant performance gains on various NLP tasks. Prompt engineering requires composing natural language instructions called prompts to elicit knowledge from LLMs in a structured way. Unlike previous state-of-the-art (SoTA) models, prompt engineering does not require extensive parameter re-training or fine-tuning based on the given NLP task and thus solely operates on the embedded knowledge of LLMs. Additionally, LLM enthusiasts can intelligently extract LLMs' knowledge through a basic natural language conversational exchange or prompt engineering, allowing more and more people even without deep mathematical machine learning background to experiment with LLMs. With prompt engineering gaining popularity in the last two years, researchers have come up with numerous engineering techniques around designing prompts to improve accuracy of information extraction from the LLMs. In this paper, we summarize different prompting techniques and club them together based on different NLP tasks that they have been used for. We further granularly highlight the performance of these prompting strategies on various datasets belonging to that NLP task, talk about the corresponding LLMs used, present a taxonomy diagram and discuss the possible SoTA for specific datasets. In total, we read and present a survey of 44 research papers which talk about 39 different prompting methods on 29 different NLP tasks of which most of them have been published in the last two years.

A Survey of Prompt Engineering Methods in Large Language Models for Different NLP Tasks

TL;DR

This survey addresses the problem of systematically understanding how prompting strategies enable LLMs to perform a wide range of NLP tasks without fine-tuning. It organizes 39 prompting methods into a comprehensive taxonomy and maps them to 29 NLP tasks, drawing on 44 papers to extract overall trends, insights, and potential SoTA approaches for each dataset. The key contributions include a detailed taxonomy diagram, dataset-level performance discussions, and a standardized view of prompt techniques across tasks, highlighting the growing importance of structured prompts, reasoning traces, and retrieval methods. The work's practical impact lies in guiding researchers and practitioners to select effective prompting strategies for specific tasks, datasets, and LLMs, while identifying gaps and directions for future research in prompt engineering.

Abstract

Large language models (LLMs) have shown remarkable performance on many different Natural Language Processing (NLP) tasks. Prompt engineering plays a key role in adding more to the already existing abilities of LLMs to achieve significant performance gains on various NLP tasks. Prompt engineering requires composing natural language instructions called prompts to elicit knowledge from LLMs in a structured way. Unlike previous state-of-the-art (SoTA) models, prompt engineering does not require extensive parameter re-training or fine-tuning based on the given NLP task and thus solely operates on the embedded knowledge of LLMs. Additionally, LLM enthusiasts can intelligently extract LLMs' knowledge through a basic natural language conversational exchange or prompt engineering, allowing more and more people even without deep mathematical machine learning background to experiment with LLMs. With prompt engineering gaining popularity in the last two years, researchers have come up with numerous engineering techniques around designing prompts to improve accuracy of information extraction from the LLMs. In this paper, we summarize different prompting techniques and club them together based on different NLP tasks that they have been used for. We further granularly highlight the performance of these prompting strategies on various datasets belonging to that NLP task, talk about the corresponding LLMs used, present a taxonomy diagram and discuss the possible SoTA for specific datasets. In total, we read and present a survey of 44 research papers which talk about 39 different prompting methods on 29 different NLP tasks of which most of them have been published in the last two years.
Paper Structure (72 sections, 1 figure)

This paper contains 72 sections, 1 figure.

Table of Contents

  1. Introduction
  2. Prompt Engineering Techniques
  3. Basic/Standard/Vanilla Prompting
  4. Chain-of-Thought (CoT)
  5. Self-Consistency
  6. Ensemble Refinement (ER)
  7. Automatic Chain-of-Thought (Auto-CoT)
  8. Complex CoT
  9. Program-of-Thoughts (PoT)
  10. Least-to-Most
  11. Chain-of-Symbol (CoS)
  12. Structured Chain-of-Thought (SCoT)
  13. Plan-and-Solve (PS)
  14. MathPrompter
  15. Contrastive CoT/ Contrastive Self-Consistency
  16. ...and 57 more sections

Figures (1)

  • Figure 1: Taxonomy Diagram of Prompt Engineering Methods Across Different NLP Tasks