CEC-Zero: Zero-Supervision Character Error Correction with Self-Generated Rewards

TL;DR

CEC‑Zero introduces a zero‑supervision reinforcement learning framework for Chinese spelling correction by synthesizing errorful inputs from clean text and using a cluster–consensus reward to guide self‑generated corrections. The method formalizes the task, constructs pseudo‑labelled data with a perturbation library, and optimizes a policy via PPO, achieving unbiased learning signals and provable convergence. Empirically, it delivers 10–13 F1 gains over supervised baselines and 5–8 F1 gains over strong LLM fine‑tunes across nine benchmarks, with 32B RL reaching 68.2% sentence‑level F1 on CSCD‑NS; robustness improves notably on visually noisy and multi‑error texts, and scaling analyses indicate RL‑driven gains beyond a certain model size. This label‑free approach enables robust, scalable CSC in real‑world noisy text pipelines, unlocking broader potential for LLMs in domain‑shifted, error‑prone settings.

Abstract

Large-scale Chinese spelling correction (CSC) remains critical for real-world text processing, yet existing LLMs and supervised methods lack robustness to novel errors and rely on costly annotations. We introduce CEC-Zero, a zero-supervision reinforcement learning framework that addresses this by enabling LLMs to correct their own mistakes. CEC-Zero synthesizes errorful inputs from clean text, computes cluster-consensus rewards via semantic similarity and candidate agreement, and optimizes the policy with PPO. It outperforms supervised baselines by 10--13 F$_1$ points and strong LLM fine-tunes by 5--8 points across 9 benchmarks, with theoretical guarantees of unbiased rewards and convergence. CEC-Zero establishes a label-free paradigm for robust, scalable CSC, unlocking LLM potential in noisy text pipelines.

Figures (9)

• Figure 1: Three routes to Chinese spelling correction. BERT taggers rely on token‑level labels and can only perform one‑to‑one glyph swaps, while existing LLM-based methods train on the same pairs with sentence‑level MLE yet still learns by teacher forcing. CEC‑Zero instead self‑perturbs raw sentences and optimises with PPO, yielding robust label‑free correction.
• Figure 2: CEC‑Zero framework. Clean sentences are synthetically perturbed to create unlimited $(\mathbf{x},\mathbf{y})$ pairs; an LLM, post‑trained with self‑play PPO, produces multiple candidate fixes whose cluster–consensus reward blends (i) pairwise similarity to the clean reference and (ii) mutual agreement among candidates, enabling robust Chinese spelling correction without any human labels.
• Figure 3: Error taxonomy for the CS benchmark.
• Figure 4: Sentence‑level F1 (%) on CS grouped by the number of distinct error tokens.
• Figure 5: Ablation study of different reward variants.

Theorems & Definitions (13)

• Lemma 1: Exactness
• proof
• Corollary 1: Low variance
• Theorem 1: Algorithm‑specific non‑asymptotic rate
• proof : Proof sketch
• Theorem 2: Uniform convergence
• proof : Proof sketch
• Lemma 2: Exactness
• Corollary 2: Low variance
• Theorem 3: Non-asymptotic convergence