Loss-Aware Curriculum Learning for Chinese Grammatical Error Correction

TL;DR

This work targets Chinese Grammatical Error Correction (CGEC) by addressing the uneven difficulty of training samples. It introduces a loss-aware, multi-granularity Curriculum Learning framework consisting of Batch-Level data loading guided by cross-entropy-based difficulty and Instance-Level adaptive loss weighting via Monte Carlo dropout, enabling the model to focus on harder corrections. The approach is validated by applying it to multiple pre-trained language models (e.g., BART, mT5, SynGEC) and testing on NLPCC and MuCGEC datasets, where it consistently outperforms strong baselines. The results suggest that aligning training with instance-level difficulty enhances CGEC performance and generalizes across PLMs, offering a practical method to improve grammatical-error correction systems.

Abstract

Chinese grammatical error correction (CGEC) aims to detect and correct errors in the input Chinese sentences. Recently, Pre-trained Language Models (PLMS) have been employed to improve the performance. However, current approaches ignore that correction difficulty varies across different instances and treat these samples equally, enhancing the challenge of model learning. To address this problem, we propose a multi-granularity Curriculum Learning (CL) framework. Specifically, we first calculate the correction difficulty of these samples and feed them into the model from easy to hard batch by batch. Then Instance-Level CL is employed to help the model optimize in the appropriate direction automatically by regulating the loss function. Extensive experimental results and comprehensive analyses of various datasets prove the effectiveness of our method.

Figures (2)

• Figure 1: An Overview of Loss-Aware Curriculum Learning (CL) framework for the CGEC task.
• Figure 2: Improvements of $F_{0.5}$ at different difficulty intervals on NLPCC-test.