Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Structured Literature Review on Traditional Approaches in Current Natural Language Processing

Robin Jegan, Andreas Henrich

TL;DR

This paper surveys whether traditional NLP techniques persist amid the dominance of neural models and LLMs, focusing on five application areas: classification, information extraction, relation extraction, text simplification, and text summarization. It defines traditional methods and analyzes ACM DL publications from 2023 (with some 2015-2024 coverage) to identify how these techniques are used as pipelines, baselines, or core methods. The findings show traditional models still appear across all five scenarios, with text summarization showing the highest share of traditional approaches and classification frequently using SVM/TF-IDF baselines. The results imply that traditional techniques offer advantages in reproducibility, efficiency, and explainability, and the study outlines criteria for when to employ them in future NLP work.

Abstract

The continued rise of neural networks and large language models in the more recent past has altered the natural language processing landscape, enabling new approaches towards typical language tasks and achieving mainstream success. Despite the huge success of large language models, many disadvantages still remain and through this work we assess the state of the art in five application scenarios with a particular focus on the future perspectives and sensible application scenarios of traditional and older approaches and techniques. In this paper we survey recent publications in the application scenarios classification, information and relation extraction, text simplification as well as text summarization. After defining our terminology, i.e., which features are characteristic for traditional techniques in our interpretation for the five scenarios, we survey if such traditional approaches are still being used, and if so, in what way they are used. It turns out that all five application scenarios still exhibit traditional models in one way or another, as part of a processing pipeline, as a comparison/baseline to the core model of the respective paper, or as the main model(s) of the paper. For the complete statistics, see https://zenodo.org/records/13683801

A Structured Literature Review on Traditional Approaches in Current Natural Language Processing

TL;DR

This paper surveys whether traditional NLP techniques persist amid the dominance of neural models and LLMs, focusing on five application areas: classification, information extraction, relation extraction, text simplification, and text summarization. It defines traditional methods and analyzes ACM DL publications from 2023 (with some 2015-2024 coverage) to identify how these techniques are used as pipelines, baselines, or core methods. The findings show traditional models still appear across all five scenarios, with text summarization showing the highest share of traditional approaches and classification frequently using SVM/TF-IDF baselines. The results imply that traditional techniques offer advantages in reproducibility, efficiency, and explainability, and the study outlines criteria for when to employ them in future NLP work.

Abstract

The continued rise of neural networks and large language models in the more recent past has altered the natural language processing landscape, enabling new approaches towards typical language tasks and achieving mainstream success. Despite the huge success of large language models, many disadvantages still remain and through this work we assess the state of the art in five application scenarios with a particular focus on the future perspectives and sensible application scenarios of traditional and older approaches and techniques. In this paper we survey recent publications in the application scenarios classification, information and relation extraction, text simplification as well as text summarization. After defining our terminology, i.e., which features are characteristic for traditional techniques in our interpretation for the five scenarios, we survey if such traditional approaches are still being used, and if so, in what way they are used. It turns out that all five application scenarios still exhibit traditional models in one way or another, as part of a processing pipeline, as a comparison/baseline to the core model of the respective paper, or as the main model(s) of the paper. For the complete statistics, see https://zenodo.org/records/13683801
Paper Structure (13 sections, 1 figure, 2 tables)

This paper contains 13 sections, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Classification
  4. Information Extraction
  5. Relation Extraction
  6. Text Simplification
  7. Text Summarization
  8. Terminology
  9. Structured Literature Review
  10. Current Work on Traditional Approaches
  11. Application Scenarios for Traditional Approaches
  12. Conclusion and Future Work
  13. Limitations

Figures (1)

  • Figure 1: Results of the SLR, "Traditional Papers" stands for the number of papers using traditional approaches in some way.