Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Pre-Trained Language Models for Keyphrase Prediction: A Review

Muhammad Umair, Tangina Sultana, Young-Koo Lee

TL;DR

This paper extensively examines the topic of pre-trained language models for keyphrase prediction (PLM-KP), which are trained on large text corpora via different learning techniques to provide respective insights into these two types of tasks in NLP, precisely, Keyphrase Extraction (KPE) and Keyphrase Generation (KPG).

Abstract

Keyphrase Prediction (KP) is essential for identifying keyphrases in a document that can summarize its content. However, recent Natural Language Processing (NLP) advances have developed more efficient KP models using deep learning techniques. The limitation of a comprehensive exploration jointly both keyphrase extraction and generation using pre-trained language models spotlights a critical gap in the literature, compelling our survey paper to bridge this deficiency and offer a unified and in-depth analysis to address limitations in previous surveys. This paper extensively examines the topic of pre-trained language models for keyphrase prediction (PLM-KP), which are trained on large text corpora via different learning (supervisor, unsupervised, semi-supervised, and self-supervised) techniques, to provide respective insights into these two types of tasks in NLP, precisely, Keyphrase Extraction (KPE) and Keyphrase Generation (KPG). We introduce appropriate taxonomies for PLM-KPE and KPG to highlight these two main tasks of NLP. Moreover, we point out some promising future directions for predicting keyphrases.

Pre-Trained Language Models for Keyphrase Prediction: A Review

TL;DR

This paper extensively examines the topic of pre-trained language models for keyphrase prediction (PLM-KP), which are trained on large text corpora via different learning techniques to provide respective insights into these two types of tasks in NLP, precisely, Keyphrase Extraction (KPE) and Keyphrase Generation (KPG).

Abstract

Keyphrase Prediction (KP) is essential for identifying keyphrases in a document that can summarize its content. However, recent Natural Language Processing (NLP) advances have developed more efficient KP models using deep learning techniques. The limitation of a comprehensive exploration jointly both keyphrase extraction and generation using pre-trained language models spotlights a critical gap in the literature, compelling our survey paper to bridge this deficiency and offer a unified and in-depth analysis to address limitations in previous surveys. This paper extensively examines the topic of pre-trained language models for keyphrase prediction (PLM-KP), which are trained on large text corpora via different learning (supervisor, unsupervised, semi-supervised, and self-supervised) techniques, to provide respective insights into these two types of tasks in NLP, precisely, Keyphrase Extraction (KPE) and Keyphrase Generation (KPG). We introduce appropriate taxonomies for PLM-KPE and KPG to highlight these two main tasks of NLP. Moreover, we point out some promising future directions for predicting keyphrases.
Paper Structure (32 sections, 4 figures, 9 tables)

This paper contains 32 sections, 4 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Preliminary
  3. Previous Surveys
  4. Research criteria
  5. Background and Summaries of LLMs
  6. Background
  7. Summaries of Large Language Models
  8. PLMs for KPE
  9. Attention Mechanism
  10. Graph-based Ranking
  11. Semantic Importance
  12. Phrase-Documents Similarity
  13. Multimodal Keyphrase Extration and Generaion
  14. PLMs for KPG
  15. Low-resource
  16. ...and 17 more sections

Figures (4)

  • Figure 1: Overview of Keyphrase Prediction using Pre-trained Language Models for Keyphrase Extraction and Keyphrase Generation.
  • Figure 2: Categorization of Keyphrase Prediction using Pre-trained Language Models (PLM-KP), centered around two fundamental NLP tasks: Pre-trained Language Model Keyphrase Extraction (PLM-KPE) and Pre-trained Language Model Keyphrase Generation (PLM-KPG).
  • Figure 3: Categorization of Low-resource
  • Figure 4: Categorization of Domain-specific