Natural Language Processing with Commonsense Knowledge: A Survey

TL;DR

The challenges and emerging trends in enhancing NLP systems with commonsense reasoning are examined, and key methodologies for incorporating commonsense knowledge and their applications across different NLP tasks are highlighted.

Abstract

Commonsense knowledge is essential for advancing natural language processing (NLP) by enabling models to engage in human-like reasoning, which requires a deeper understanding of context and often involves making inferences based on implicit external knowledge. This paper explores the integration of commonsense knowledge into various NLP tasks. We begin by reviewing prominent commonsense knowledge bases and then discuss the benchmarks used to evaluate the commonsense reasoning capabilities of NLP models, particularly language models. Furthermore, we highlight key methodologies for incorporating commonsense knowledge and their applications across different NLP tasks. The paper also examines the challenges and emerging trends in enhancing NLP systems with commonsense reasoning. All literature referenced in this survey can be accessed via our GitHub repository: https://github.com/yuboxie/awesome-commonsense.

Figures (3)

• Figure 1: Categorization of commonsense reasoning benchmarks by task type.
• Figure 2: Overview of commonsense reasoning benchmarks categorized by the types of commonsense knowledge they evaluate.
• Figure 3: Three different ways for NLP models to incorporate and enhance commonsense knowledge: (a) External Memorization; (b) Global Optimization; (c) Local Modification.