Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enriching the Korean Learner Corpus with Multi-reference Annotations and Rubric-Based Scoring

Jayoung Song, KyungTae Lim, Jungyeul Park

TL;DR

The paper addresses the scarcity of Korean L2 writing resources by enriching the KoLLA corpus with (i) multi-reference grammatical error correction (GEC) annotations and (ii) rubric-based scoring aligned with the Korean National Language Institute. It introduces a multi-reference GEC dataset that captures linguistic variability through two corrections per sentence and expands the error typology to Korean-specific phenomena, including word-boundary adjustments and postposition/morphology errors. It also adds rubric-based automated essay scoring data, with rigorously trained annotators and high inter-annotator reliability (overall Cohen’s κ ≈ 0.8241), across four learner groups. Together, these enhancements yield a robust resource for evaluating GEC systems and for driving automated writing assessment and pedagogy in Korean L2 education, with the enriched KoLLA released under GPLv3 for public use.

Abstract

Despite growing global interest in Korean language education, there remains a significant lack of learner corpora tailored to Korean L2 writing. To address this gap, we enhance the KoLLA Korean learner corpus by adding multiple grammatical error correction (GEC) references, thereby enabling more nuanced and flexible evaluation of GEC systems, and reflects the variability of human language. Additionally, we enrich the corpus with rubric-based scores aligned with guidelines from the Korean National Language Institute, capturing grammatical accuracy, coherence, and lexical diversity. These enhancements make KoLLA a robust and standardized resource for research in Korean L2 education, supporting advancements in language learning, assessment, and automated error correction.

Enriching the Korean Learner Corpus with Multi-reference Annotations and Rubric-Based Scoring

TL;DR

The paper addresses the scarcity of Korean L2 writing resources by enriching the KoLLA corpus with (i) multi-reference grammatical error correction (GEC) annotations and (ii) rubric-based scoring aligned with the Korean National Language Institute. It introduces a multi-reference GEC dataset that captures linguistic variability through two corrections per sentence and expands the error typology to Korean-specific phenomena, including word-boundary adjustments and postposition/morphology errors. It also adds rubric-based automated essay scoring data, with rigorously trained annotators and high inter-annotator reliability (overall Cohen’s κ ≈ 0.8241), across four learner groups. Together, these enhancements yield a robust resource for evaluating GEC systems and for driving automated writing assessment and pedagogy in Korean L2 education, with the enriched KoLLA released under GPLv3 for public use.

Abstract

Despite growing global interest in Korean language education, there remains a significant lack of learner corpora tailored to Korean L2 writing. To address this gap, we enhance the KoLLA Korean learner corpus by adding multiple grammatical error correction (GEC) references, thereby enabling more nuanced and flexible evaluation of GEC systems, and reflects the variability of human language. Additionally, we enrich the corpus with rubric-based scores aligned with guidelines from the Korean National Language Institute, capturing grammatical accuracy, coherence, and lexical diversity. These enhancements make KoLLA a robust and standardized resource for research in Korean L2 education, supporting advancements in language learning, assessment, and automated error correction.
Paper Structure (13 sections, 1 figure, 2 tables, 1 algorithm)

This paper contains 13 sections, 1 figure, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Previous Works
  3. Development and Annotation
  4. Introduction to KoLLA
  5. Multi-reference GEC corpus
  6. Rubric-based scoring dataset
  7. Findings and Discussion
  8. Multi-reference grammatical error annotation
  9. Rubric-based scoring for writing assessment
  10. Implications and Future Directions
  11. Summary of findings
  12. Conclusion
  13. Dataset Description

Figures (1)

  • Figure 1: Examples from the Korean M2 file: Annotator 0 corrected the sentence as I didn’t eat the airplane food, while Annotator 1 corrected it as The airplane food didn’t agree with me, resulting in divergent annotation outcomes. For illustrative purposes, index numbers are included to refer to word positions in the source sentence. These indices are not present in the original M2 files but are added here to facilitate discussion, comparison, and reference.