Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Mask of Civility: Benchmarking Chinese Mock Politeness Comprehension in Large Language Models

Yitong Zhang, Yuhan Xiang, Mingxuan Liu

TL;DR

This study benchmarks Chinese pragmatic understanding in large language models by classifying politeness, impoliteness, and mock politeness across authentic and simulated discourse using six LLMs under four prompting strategies. Grounded in Rapport Management Theory and Mock Politeness, it demonstrates that knowledge-enhanced and hybrid prompting substantially improve recognition, with Chinese-developed models performing better on Chinese pragmatics. Key contributions include a three-category dataset, a comparative evaluation of prompting strategies, and detailed error analysis highlighting the difficulty of detecting mock politeness. The findings have practical implications for building more context-aware, human-centered AI systems and advance the interdisciplinary integration of pragmatic theory with language technology.

Abstract

From a pragmatic perspective, this study systematically evaluates the differences in performance among representative large language models (LLMs) in recognizing politeness, impoliteness, and mock politeness phenomena in Chinese. Addressing the existing gaps in pragmatic comprehension, the research adopts the frameworks of Rapport Management Theory and the Model of Mock Politeness to construct a three-category dataset combining authentic and simulated Chinese discourse. Six representative models, including GPT-5.1 and DeepSeek, were selected as test subjects and evaluated under four prompting conditions: zero-shot, few-shot, knowledge-enhanced, and hybrid strategies. This study serves as a meaningful attempt within the paradigm of ``Great Linguistics,'' offering a novel approach to applying pragmatic theory in the age of technological transformation. It also responds to the contemporary question of how technology and the humanities may coexist, representing an interdisciplinary endeavor that bridges linguistic technology and humanistic reflection.

The Mask of Civility: Benchmarking Chinese Mock Politeness Comprehension in Large Language Models

TL;DR

This study benchmarks Chinese pragmatic understanding in large language models by classifying politeness, impoliteness, and mock politeness across authentic and simulated discourse using six LLMs under four prompting strategies. Grounded in Rapport Management Theory and Mock Politeness, it demonstrates that knowledge-enhanced and hybrid prompting substantially improve recognition, with Chinese-developed models performing better on Chinese pragmatics. Key contributions include a three-category dataset, a comparative evaluation of prompting strategies, and detailed error analysis highlighting the difficulty of detecting mock politeness. The findings have practical implications for building more context-aware, human-centered AI systems and advance the interdisciplinary integration of pragmatic theory with language technology.

Abstract

From a pragmatic perspective, this study systematically evaluates the differences in performance among representative large language models (LLMs) in recognizing politeness, impoliteness, and mock politeness phenomena in Chinese. Addressing the existing gaps in pragmatic comprehension, the research adopts the frameworks of Rapport Management Theory and the Model of Mock Politeness to construct a three-category dataset combining authentic and simulated Chinese discourse. Six representative models, including GPT-5.1 and DeepSeek, were selected as test subjects and evaluated under four prompting conditions: zero-shot, few-shot, knowledge-enhanced, and hybrid strategies. This study serves as a meaningful attempt within the paradigm of ``Great Linguistics,'' offering a novel approach to applying pragmatic theory in the age of technological transformation. It also responds to the contemporary question of how technology and the humanities may coexist, representing an interdisciplinary endeavor that bridges linguistic technology and humanistic reflection.
Paper Structure (14 sections, 1 figure, 5 tables)

This paper contains 14 sections, 1 figure, 5 tables.

Table of Contents

  1. Introduction
  2. Background and Theory
  3. Definitions and Frameworks
  4. Current Status of LLM Pragmatics
  5. Methodology
  6. Dataset Construction
  7. Model Selection
  8. Experimental Design
  9. Results and Analysis
  10. Overall Performance
  11. Analysis by Strategy
  12. Error Analysis
  13. Conclusion
  14. Dataset Examples

Figures (1)

  • Figure 1: Confusion Matrix for Kimi-k2-0905-preview across four prompting strategies. The heatmap illustrates the distribution of predicted labels against the ground truth for each experimental setting.