Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Organic Data-Driven Approach for Turkish Grammatical Error Correction and LLMs

Asım Ersoy, Olcay Taner Yıldız

TL;DR

Turkish Grammatical Error Correction (GEC) data suffer from scarcity and noise in web corpora. The authors introduce clean insertions, an organic data-driven method that builds parallel Turkish GEC data from unrestricted text using a manually crafted spelling dictionary, a Deasciifier, and a Spell Checker, without requiring clean seeds. They create OSCAR GEC (≈2.3M sentences) and GPT GEC (≈100k sentence pairs) and evaluate against GECTurk across ERRANT-TR metrics, achieving state-of-the-art on two of three public test sets and showing data cleaning can reduce language model training losses. By fine-tuning mT5 and conducting GPT-2 experiments, they demonstrate practical benefits in LM training and release multiple datasets and models, highlighting the approach’s potential to improve Turkish GEC and downstream LM performance.

Abstract

Grammatical Error Correction has seen significant progress with the recent advancements in deep learning. As those methods require huge amounts of data, synthetic datasets are being built to fill this gap. Unfortunately, synthetic datasets are not organic enough in some cases and even require clean data to start with. Furthermore, most of the work that has been done is focused mostly on English. In this work, we introduce a new organic data-driven approach, clean insertions, to build parallel Turkish Grammatical Error Correction datasets from any organic data, and to clean the data used for training Large Language Models. We achieve state-of-the-art results on two Turkish Grammatical Error Correction test sets out of the three publicly available ones. We also show the effectiveness of our method on the training losses of training language models.

Organic Data-Driven Approach for Turkish Grammatical Error Correction and LLMs

TL;DR

Turkish Grammatical Error Correction (GEC) data suffer from scarcity and noise in web corpora. The authors introduce clean insertions, an organic data-driven method that builds parallel Turkish GEC data from unrestricted text using a manually crafted spelling dictionary, a Deasciifier, and a Spell Checker, without requiring clean seeds. They create OSCAR GEC (≈2.3M sentences) and GPT GEC (≈100k sentence pairs) and evaluate against GECTurk across ERRANT-TR metrics, achieving state-of-the-art on two of three public test sets and showing data cleaning can reduce language model training losses. By fine-tuning mT5 and conducting GPT-2 experiments, they demonstrate practical benefits in LM training and release multiple datasets and models, highlighting the approach’s potential to improve Turkish GEC and downstream LM performance.

Abstract

Grammatical Error Correction has seen significant progress with the recent advancements in deep learning. As those methods require huge amounts of data, synthetic datasets are being built to fill this gap. Unfortunately, synthetic datasets are not organic enough in some cases and even require clean data to start with. Furthermore, most of the work that has been done is focused mostly on English. In this work, we introduce a new organic data-driven approach, clean insertions, to build parallel Turkish Grammatical Error Correction datasets from any organic data, and to clean the data used for training Large Language Models. We achieve state-of-the-art results on two Turkish Grammatical Error Correction test sets out of the three publicly available ones. We also show the effectiveness of our method on the training losses of training language models.
Paper Structure (23 sections, 2 figures, 9 tables)

This paper contains 23 sections, 2 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Datasets
  4. Approaches
  5. Turkish Grammatical Error Correction
  6. Data Methodology
  7. OSCAR GEC
  8. Deasciifier
  9. Spell Checker
  10. GPT GEC
  11. Datasets Overview
  12. Experimental Setup
  13. Models
  14. Evaluation
  15. Results
  16. ...and 8 more sections

Figures (2)

  • Figure 1: A pipeline of the creation process of OSCAR GEC showing all the steps involved in creating the OSCAR GEC dataset
  • Figure 2: One example from the Turkish Tweets and the output of the three models OSCAR GEC, GPT GEC, and Sequence Tagger. The red segments are incorrect and the green ones are correct.