Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CSCD-NS: a Chinese Spelling Check Dataset for Native Speakers

Yong Hu, Fandong Meng, Jie Zhou

TL;DR

CSCD-NS provides the largest native-speaker Chinese spelling check dataset to date, sourced from authentic Weibo text and totaling 40,000 manually annotated samples. It reveals a higher prevalence of word-level and phonetic errors among native speakers, facilitated by a phonetic-semantic tagging scheme and comprehensive statistical analyses. To address data scarcity, the authors introduce an IME-based pseudo data generation method that closely matches real input error distributions and yields notable gains when used for pretraining. Across diverse models, results show that BERT-like token-level classifiers outperform generative approaches under the task's equal-length and pronunciation constraints, highlighting both the potential and the still substantial room for improvement in native-speaker CSC. These resources and findings aim to spur further research into robust, language-model-based spelling correction for native Chinese users.

Abstract

In this paper, we present CSCD-NS, the first Chinese spelling check (CSC) dataset designed for native speakers, containing 40,000 samples from a Chinese social platform. Compared with existing CSC datasets aimed at Chinese learners, CSCD-NS is ten times larger in scale and exhibits a distinct error distribution, with a significantly higher proportion of word-level errors. To further enhance the data resource, we propose a novel method that simulates the input process through an input method, generating large-scale and high-quality pseudo data that closely resembles the actual error distribution and outperforms existing methods. Moreover, we investigate the performance of various models in this scenario, including large language models (LLMs), such as ChatGPT. The result indicates that generative models underperform BERT-like classification models due to strict length and pronunciation constraints. The high prevalence of word-level errors also makes CSC for native speakers challenging enough, leaving substantial room for improvement.

CSCD-NS: a Chinese Spelling Check Dataset for Native Speakers

TL;DR

CSCD-NS provides the largest native-speaker Chinese spelling check dataset to date, sourced from authentic Weibo text and totaling 40,000 manually annotated samples. It reveals a higher prevalence of word-level and phonetic errors among native speakers, facilitated by a phonetic-semantic tagging scheme and comprehensive statistical analyses. To address data scarcity, the authors introduce an IME-based pseudo data generation method that closely matches real input error distributions and yields notable gains when used for pretraining. Across diverse models, results show that BERT-like token-level classifiers outperform generative approaches under the task's equal-length and pronunciation constraints, highlighting both the potential and the still substantial room for improvement in native-speaker CSC. These resources and findings aim to spur further research into robust, language-model-based spelling correction for native Chinese users.

Abstract

In this paper, we present CSCD-NS, the first Chinese spelling check (CSC) dataset designed for native speakers, containing 40,000 samples from a Chinese social platform. Compared with existing CSC datasets aimed at Chinese learners, CSCD-NS is ten times larger in scale and exhibits a distinct error distribution, with a significantly higher proportion of word-level errors. To further enhance the data resource, we propose a novel method that simulates the input process through an input method, generating large-scale and high-quality pseudo data that closely resembles the actual error distribution and outperforms existing methods. Moreover, we investigate the performance of various models in this scenario, including large language models (LLMs), such as ChatGPT. The result indicates that generative models underperform BERT-like classification models due to strict length and pronunciation constraints. The high prevalence of word-level errors also makes CSC for native speakers challenging enough, leaving substantial room for improvement.
Paper Structure (29 sections, 4 equations, 5 figures, 15 tables)

This paper contains 29 sections, 4 equations, 5 figures, 15 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. CSCD-NS
  4. Data Source
  5. Data Selection
  6. Data Annotation
  7. Analysis on Basic Statistics
  8. Analysis on Error Distribution
  9. Data Augmentation
  10. Data Preparation
  11. IME-based Pseudo Data Generation
  12. LCSTS-IME-2M
  13. Experiments
  14. Basic Settings
  15. Main Results
  16. ...and 14 more sections

Figures (5)

  • Figure 1: An error from SIGHAN: misspelling “错误” as “错勿”. Despite having the same pronunciation, it's hard to reproduce this error in the given context through a Chinese IME, no matter what input form is used.
  • Figure 2: An authentic Weibo post from LCSTS, where the phrase "效力于" is mistakenly written as "效力与".
  • Figure 3: The comparison of error distribution (%) at phonetic level (above) and semantic level (below).
  • Figure 4: The IME-based pseudo data generation process.
  • Figure 5: The comparison of error distribution (%) at phonetic level (above) and semantic level (below).