Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MCTS: A Multi-Reference Chinese Text Simplification Dataset

Ruining Chong, Luming Lu, Liner Yang, Jinran Nie, Zhenghao Liu, Shuo Wang, Shuhan Zhou, Yaoxin Li, Erhong Yang

TL;DR

This paper introduces MCTS, the first large-scale, multi-reference Chinese text simplification dataset, addressing the longstanding lack of evaluation data for Chinese. By sourcing 723 CTB-origin sentences and manually annotating 3,615 simplifications, the authors provide a diverse benchmark that captures multiple rewriting transformations, including paraphrase, compression, and structure changes. They also release a substantial cross-lingual training resource (approximately 691,474 Chinese sentence pairs) derived through translation and English simplification, enabling training of Chinese models without relying on large native parallel corpora. Through extensive experiments with unsupervised baselines and large language models (including GPT-3.5-turbo and text-davinci-003), the study shows that while LLMs surpass traditional unsupervised methods, there remains a gap relative to human-authored simplifications. Overall, MCTS offers a rigorous evaluation framework and practical data resources that can guide future Chinese text simplification research and model development.

Abstract

Text simplification aims to make the text easier to understand by applying rewriting transformations. There has been very little research on Chinese text simplification for a long time. The lack of generic evaluation data is an essential reason for this phenomenon. In this paper, we introduce MCTS, a multi-reference Chinese text simplification dataset. We describe the annotation process of the dataset and provide a detailed analysis. Furthermore, we evaluate the performance of several unsupervised methods and advanced large language models. We additionally provide Chinese text simplification parallel data that can be used for training, acquired by utilizing machine translation and English text simplification. We hope to build a basic understanding of Chinese text simplification through the foundational work and provide references for future research. All of the code and data are released at https://github.com/blcuicall/mcts/.

MCTS: A Multi-Reference Chinese Text Simplification Dataset

TL;DR

This paper introduces MCTS, the first large-scale, multi-reference Chinese text simplification dataset, addressing the longstanding lack of evaluation data for Chinese. By sourcing 723 CTB-origin sentences and manually annotating 3,615 simplifications, the authors provide a diverse benchmark that captures multiple rewriting transformations, including paraphrase, compression, and structure changes. They also release a substantial cross-lingual training resource (approximately 691,474 Chinese sentence pairs) derived through translation and English simplification, enabling training of Chinese models without relying on large native parallel corpora. Through extensive experiments with unsupervised baselines and large language models (including GPT-3.5-turbo and text-davinci-003), the study shows that while LLMs surpass traditional unsupervised methods, there remains a gap relative to human-authored simplifications. Overall, MCTS offers a rigorous evaluation framework and practical data resources that can guide future Chinese text simplification research and model development.

Abstract

Text simplification aims to make the text easier to understand by applying rewriting transformations. There has been very little research on Chinese text simplification for a long time. The lack of generic evaluation data is an essential reason for this phenomenon. In this paper, we introduce MCTS, a multi-reference Chinese text simplification dataset. We describe the annotation process of the dataset and provide a detailed analysis. Furthermore, we evaluate the performance of several unsupervised methods and advanced large language models. We additionally provide Chinese text simplification parallel data that can be used for training, acquired by utilizing machine translation and English text simplification. We hope to build a basic understanding of Chinese text simplification through the foundational work and provide references for future research. All of the code and data are released at https://github.com/blcuicall/mcts/.
Paper Structure (33 sections, 2 figures, 6 tables)

This paper contains 33 sections, 2 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Evaluation Data for English Text Simplification
  4. Unsupervised Text Simplification
  5. Creating MCTS
  6. Data Preparation
  7. Annotation Process
  8. Annotator Recruitment
  9. Simplification Instructions
  10. Qualification Test
  11. Annotation Round
  12. Dataset Analysis
  13. Text Features
  14. Results and Analysis
  15. Comparison with CSS
  16. ...and 18 more sections

Figures (2)

  • Figure 1: Density of text features in simplifications from MCTS
  • Figure 2: Pseudo data acquisition process