A School Student Essay Corpus for Analyzing Interactions of Argumentative Structure and Quality

TL;DR

The paper tackles the lack of school-student argumentative writing data with ground-truth quality annotations by introducing a German corpus of 1,320 essays annotated for four levels of argumentative structure and five essay-quality aspects. It provides a detailed annotation scheme, reports high inter-annotator reliability, and analyzes interactions between structure and quality. Through transformer-based baselines and AdapterFusion, the authors show that incorporating multi-level argumentative structure improves automatic essay scoring, enabling more targeted quality-oriented writing feedback. This resource and the accompanying baselines pave the way for developing robust, feedback-driven writing-support tools tailored to school students.

Abstract

Learning argumentative writing is challenging. Besides writing fundamentals such as syntax and grammar, learners must select and arrange argument components meaningfully to create high-quality essays. To support argumentative writing computationally, one step is to mine the argumentative structure. When combined with automatic essay scoring, interactions of the argumentative structure and quality scores can be exploited for comprehensive writing support. Although studies have shown the usefulness of using information about the argumentative structure for essay scoring, no argument mining corpus with ground-truth essay quality annotations has been published yet. Moreover, none of the existing corpora contain essays written by school students specifically. To fill this research gap, we present a German corpus of 1,320 essays from school students of two age groups. Each essay has been manually annotated for argumentative structure and quality on multiple levels of granularity. We propose baseline approaches to argument mining and essay scoring, and we analyze interactions between both tasks, thereby laying the ground for quality-oriented argumentative writing support.

Figures (6)

• Figure 1: Exemplary annotated school student essay on the use of school funding, taken from our corpus. The text is from the FD-LEX corpus becker-mrotzek-grabrowski-2018-textkorpus, translated from German for display.
• Figure 2: Proposed annotation scheme for argumentative school student essays: Four levels of argumentative macro and micro structure (discourse functions, arguments, components, discourse modes) and five essay quality aspects.
• Figure 3: Cooccurrence matrices: Relative token-level overlap of (a) macro and micro structure and (b) component and discourse mode labels in percent. For example, 68% of all tokens labeled as Introduction on the macro level are also labeled as Topic on the micro level.
• Figure 4: AdapterFusion activation on average over the layers for each mDeBERTaV3-fusion-w/-all model per quality aspect. We average the activation for each fused adapter (discourse functions, arguments, components, discourse modes) over all instances in the most representative test set folding.
• Figure 5: Relative token-level overlap of all argumentative structure labels, seperated into the four levels of granularity. For example, 68% of all tokens labeled as Introduction are also labeled Topic.