Can Pre-trained Language Models Understand Chinese Humor?

TL;DR

This work systematically investigates whether pre-trained language models understand Chinese humor by introducing a three-step evaluation framework with four humor-related tasks and a large Chinese humor dataset. It shows that original PLMs have limited humor understanding in zero-shot settings, but fine-tuning yields substantial improvements, and linguistic knowledge—especially Chinese pinyin embeddings—significantly boosts performance when applied via explicit fusion. Interpretability analyses reveal that PLMs rely on recognizable clue words, with saliency evidence supporting partial alignment with human humor perception, though gaps remain. Downstream transfer to sentiment tasks is modest, indicating domain-specific benefits and the need for richer cultural knowledge to enhance generalization in humor understanding and generation.

Abstract

Humor understanding is an important and challenging research in natural language processing. As the popularity of pre-trained language models (PLMs), some recent work makes preliminary attempts to adopt PLMs for humor recognition and generation. However, these simple attempts do not substantially answer the question: {\em whether PLMs are capable of humor understanding?} This paper is the first work that systematically investigates the humor understanding ability of PLMs. For this purpose, a comprehensive framework with three evaluation steps and four evaluation tasks is designed. We also construct a comprehensive Chinese humor dataset, which can fully meet all the data requirements of the proposed evaluation framework. Our empirical study on the Chinese humor dataset yields some valuable observations, which are of great guiding value for future optimization of PLMs in humor understanding and generation.

Figures (4)

• Figure 1: The evaluation framework of PLMs’ humor understanding, including four tasks and three steps.
• Figure 2: The process for PLMs to perform on four humor-relevant tasks.
• Figure 3: The evaluation framework of PLMs’ humor understanding, including three steps: evaluate original PLMs (a), evaluate knowledge-enhanced PLMs (b$_1$,b$_2$), and interpret humor understanding (c).
• Figure 4: Saliency maps of some samples on the humor recognition sub-dataset based on the zero-shot learning (a$_1$) and fine-tuning (a$_2$), humor type classification sub-dataset based on the zero-shot learning (b$_1$) and fine-tuning (b$_2$). The top three salient words except [SEP] are highlighted with red boxes. We use BERT-base in these samples.