Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TiSpell: A Semi-Masked Methodology for Tibetan Spelling Correction covering Multi-Level Error with Data Augmentation

Yutong Liu, Feng Xiao, Ziyue Zhang, Yongbin Yu, Cheng Huang, Fan Gao, Xiangxiang Wang, Ma-bao Ban, Manping Fan, Thupten Tsering, Cheng Huang, Gadeng Luosang, Renzeng Duojie, Nyima Tashi

TL;DR

This work tackles multi-level Tibetan spelling correction by proposing TiSpell, an encoder-only model with a semi-masked character-level head and a syllable-level correction head. It introduces a data augmentation pipeline that generates nine corruption types from unlabeled Tibetan text to train robust correction under character-, syllable-, and mixed-level errors, optimized via a multi-task loss L = L_M + w_C L_C with w_C = 2. Evaluations on simulated and real-world data show TiSpell outperforms traditional baselines and matches or exceeds state-of-the-art results, with TiSpell-RoBERTa delivering top performance on real-world data. Ablation and interpretability analyses confirm the effectiveness of the semi-masked head, multi-head architecture, and attention-focused corrections, underscoring practical applicability for Tibetan NLP and multi-level error handling.

Abstract

Multi-level Tibetan spelling correction addresses errors at both the character and syllable levels within a unified model. Existing methods focus mainly on single-level correction and lack effective integration of both levels. Moreover, there are no open-source datasets or augmentation methods tailored for this task in Tibetan. To tackle this, we propose a data augmentation approach using unlabeled text to generate multi-level corruptions, and introduce TiSpell, a semi-masked model capable of correcting both character- and syllable-level errors. Although syllable-level correction is more challenging due to its reliance on global context, our semi-masked strategy simplifies this process. We synthesize nine types of corruptions on clean sentences to create a robust training set. Experiments on both simulated and real-world data demonstrate that TiSpell, trained on our dataset, outperforms baseline models and matches the performance of state-of-the-art approaches, confirming its effectiveness.

TiSpell: A Semi-Masked Methodology for Tibetan Spelling Correction covering Multi-Level Error with Data Augmentation

TL;DR

This work tackles multi-level Tibetan spelling correction by proposing TiSpell, an encoder-only model with a semi-masked character-level head and a syllable-level correction head. It introduces a data augmentation pipeline that generates nine corruption types from unlabeled Tibetan text to train robust correction under character-, syllable-, and mixed-level errors, optimized via a multi-task loss L = L_M + w_C L_C with w_C = 2. Evaluations on simulated and real-world data show TiSpell outperforms traditional baselines and matches or exceeds state-of-the-art results, with TiSpell-RoBERTa delivering top performance on real-world data. Ablation and interpretability analyses confirm the effectiveness of the semi-masked head, multi-head architecture, and attention-focused corrections, underscoring practical applicability for Tibetan NLP and multi-level error handling.

Abstract

Multi-level Tibetan spelling correction addresses errors at both the character and syllable levels within a unified model. Existing methods focus mainly on single-level correction and lack effective integration of both levels. Moreover, there are no open-source datasets or augmentation methods tailored for this task in Tibetan. To tackle this, we propose a data augmentation approach using unlabeled text to generate multi-level corruptions, and introduce TiSpell, a semi-masked model capable of correcting both character- and syllable-level errors. Although syllable-level correction is more challenging due to its reliance on global context, our semi-masked strategy simplifies this process. We synthesize nine types of corruptions on clean sentences to create a robust training set. Experiments on both simulated and real-world data demonstrate that TiSpell, trained on our dataset, outperforms baseline models and matches the performance of state-of-the-art approaches, confirming its effectiveness.
Paper Structure (28 sections, 11 equations, 6 figures, 7 tables, 9 algorithms)

This paper contains 28 sections, 11 equations, 6 figures, 7 tables, 9 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. Encoder-decoder Model
  4. Encoder-Only Model
  5. Detection-Correction Model
  6. Augmentations Strategies
  7. Character-Level Corruption
  8. Syllable-Level Corruption
  9. Mixed Corruption
  10. Methods
  11. Semi-Masked Character-Level Correction Head
  12. Syllable-Level Correction Head
  13. Multi-Task Training
  14. Experiments
  15. Dataset and Evaluation
  16. ...and 13 more sections

Figures (6)

  • Figure 1: The correction process of TiSpell.
  • Figure 2: The flow chart of the mixed corruption strategy. The $corrupt$ is the set of the single corruption methods.
  • Figure 3: The proposed encoder-decoder structure based on PLM.
  • Figure 4: A Prediction Example of Tispell-RoBERTa.
  • Figure 5: Visualization of Attention Score in the early and late layer in the proposed model
  • ...and 1 more figures