Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Large Language Models Meet NLP: A Survey

Libo Qin, Qiguang Chen, Xiachong Feng, Yang Wu, Yongheng Zhang, Yinghui Li, Min Li, Wanxiang Che, Philip S. Yu

TL;DR

This paper provides the first comprehensive survey of large language models in NLP by introducing a unified taxonomy that separates parameter-frozen prompting from parameter-tuning approaches. It systematically analyzes LLM applications across natural language understanding and generation tasks, mapping how zero-shot, few-shot, full-tuning, and PEFT methods perform in diverse NLP subtasks. The authors discuss future frontiers—multilingual and multimodal LLMs, tool usage, X-of-thought, hallucination, and safety—along with practical challenges and recommendations. A curated resource hub accompanies the survey to support researchers and practitioners in building effective LLM-based NLP systems.

Abstract

While large language models (LLMs) like ChatGPT have shown impressive capabilities in Natural Language Processing (NLP) tasks, a systematic investigation of their potential in this field remains largely unexplored. This study aims to address this gap by exploring the following questions: (1) How are LLMs currently applied to NLP tasks in the literature? (2) Have traditional NLP tasks already been solved with LLMs? (3) What is the future of the LLMs for NLP? To answer these questions, we take the first step to provide a comprehensive overview of LLMs in NLP. Specifically, we first introduce a unified taxonomy including (1) parameter-frozen paradigm and (2) parameter-tuning paradigm to offer a unified perspective for understanding the current progress of LLMs in NLP. Furthermore, we summarize the new frontiers and the corresponding challenges, aiming to inspire further groundbreaking advancements. We hope this work offers valuable insights into the potential and limitations of LLMs, while also serving as a practical guide for building effective LLMs in NLP.

Large Language Models Meet NLP: A Survey

TL;DR

This paper provides the first comprehensive survey of large language models in NLP by introducing a unified taxonomy that separates parameter-frozen prompting from parameter-tuning approaches. It systematically analyzes LLM applications across natural language understanding and generation tasks, mapping how zero-shot, few-shot, full-tuning, and PEFT methods perform in diverse NLP subtasks. The authors discuss future frontiers—multilingual and multimodal LLMs, tool usage, X-of-thought, hallucination, and safety—along with practical challenges and recommendations. A curated resource hub accompanies the survey to support researchers and practitioners in building effective LLM-based NLP systems.

Abstract

While large language models (LLMs) like ChatGPT have shown impressive capabilities in Natural Language Processing (NLP) tasks, a systematic investigation of their potential in this field remains largely unexplored. This study aims to address this gap by exploring the following questions: (1) How are LLMs currently applied to NLP tasks in the literature? (2) Have traditional NLP tasks already been solved with LLMs? (3) What is the future of the LLMs for NLP? To answer these questions, we take the first step to provide a comprehensive overview of LLMs in NLP. Specifically, we first introduce a unified taxonomy including (1) parameter-frozen paradigm and (2) parameter-tuning paradigm to offer a unified perspective for understanding the current progress of LLMs in NLP. Furthermore, we summarize the new frontiers and the corresponding challenges, aiming to inspire further groundbreaking advancements. We hope this work offers valuable insights into the potential and limitations of LLMs, while also serving as a practical guide for building effective LLMs in NLP.
Paper Structure (77 sections, 4 equations, 4 figures, 1 table)

This paper contains 77 sections, 4 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background
  3. Parameter-Frozen Paradigm
  4. Zero-shot Learning
  5. Few-shot Learning
  6. Parameter-Tuning Paradigm
  7. Full-parameter Tuning
  8. Parameter-efficient Tuning
  9. Comparison of Paradigms
  10. Natural Language Understanding
  11. Sentiment Analysis
  12. Parameter-Frozen Paradigm
  13. Zero-shot Learning
  14. Few-shot Learning
  15. Parameter-Tuning Paradigm
  16. ...and 62 more sections

Figures (4)

  • Figure 1: The example of applying LLMs for NLP tasks (e.g., mathematical reasoning, machine translation, information extraction and sentiment analysis).
  • Figure 2: The taxonomy of LLMs for NLP, including parameter-frozen (a) and parameter-tuning paradigm (b), where blue module with ice denotes that the parameters are kept unchanged, and orange module with fire represents the fine-tuning of full or selected parameters.
  • Figure 3: Taxonomy of LLMs for NLP including Parameter-Frozen Paradigm and Parameter-Tuning Paradigm.
  • Figure 4: The future work and new frontier for LLM in NLP tasks.