Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LAILA: A Large Trait-Based Dataset for Arabic Automated Essay Scoring

May Bashendy, Walid Massoud, Sohaila Eltanbouly, Salam Albatarni, Marwan Sayed, Abrar Abir, Houda Bouamor, Tamer Elsayed

TL;DR

LAILA introduces the first large-scale, publicly accessible Arabic AES dataset with holistic and trait-specific annotations across seven writing traits, addressing a critical data gap for Arabic language assessment. The dataset comprises 7,859 essays from 24 Qatar schools, eight prompts (five persuasive, three explanatory), and standardized CAST-based scoring with seven traits and a holistic score, validated by substantial inter-annotator agreement. The authors provide a full data collection and annotation pipeline, public release with reproducible splits (prompt-specific and cross-prompt), and baseline benchmarking across feature-based, encoder-based, and LLM models, revealing strong encoder-based baselines and notable cross-prompt generalization challenges. This resource enables transparent benchmarking, supports cross-prompt research, and accelerates the development of robust Arabic AES systems with fairness and reproducibility in mind.

Abstract

Automated Essay Scoring (AES) has gained increasing attention in recent years, yet research on Arabic AES remains limited due to the lack of publicly available datasets. To address this, we introduce LAILA, the largest publicly available Arabic AES dataset to date, comprising 7,859 essays annotated with holistic and trait-specific scores on seven dimensions: relevance, organization, vocabulary, style, development, mechanics, and grammar. We detail the dataset design, collection, and annotations, and provide benchmark results using state-of-the-art Arabic and English models in prompt-specific and cross-prompt settings. LAILA fills a critical need in Arabic AES research, supporting the development of robust scoring systems.

LAILA: A Large Trait-Based Dataset for Arabic Automated Essay Scoring

TL;DR

LAILA introduces the first large-scale, publicly accessible Arabic AES dataset with holistic and trait-specific annotations across seven writing traits, addressing a critical data gap for Arabic language assessment. The dataset comprises 7,859 essays from 24 Qatar schools, eight prompts (five persuasive, three explanatory), and standardized CAST-based scoring with seven traits and a holistic score, validated by substantial inter-annotator agreement. The authors provide a full data collection and annotation pipeline, public release with reproducible splits (prompt-specific and cross-prompt), and baseline benchmarking across feature-based, encoder-based, and LLM models, revealing strong encoder-based baselines and notable cross-prompt generalization challenges. This resource enables transparent benchmarking, supports cross-prompt research, and accelerates the development of robust Arabic AES systems with fairness and reproducibility in mind.

Abstract

Automated Essay Scoring (AES) has gained increasing attention in recent years, yet research on Arabic AES remains limited due to the lack of publicly available datasets. To address this, we introduce LAILA, the largest publicly available Arabic AES dataset to date, comprising 7,859 essays annotated with holistic and trait-specific scores on seven dimensions: relevance, organization, vocabulary, style, development, mechanics, and grammar. We detail the dataset design, collection, and annotations, and provide benchmark results using state-of-the-art Arabic and English models in prompt-specific and cross-prompt settings. LAILA fills a critical need in Arabic AES research, supporting the development of robust scoring systems.
Paper Structure (50 sections, 6 figures, 7 tables)

This paper contains 50 sections, 6 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. AES Datasets
  4. Non-Arabic AES Datasets
  5. Arabic AES Datasets
  6. Arabic AES Systems
  7. Dataset Design Principles
  8. Dataset Construction
  9. Sampling
  10. Data Collection
  11. Briefing
  12. Pre-visit Setup
  13. On-site Visits
  14. Data Cleaning
  15. Data Annotation
  16. ...and 35 more sections

Figures (6)

  • Figure 1: An overview of the construction process of LAILA dataset.
  • Figure 2: An example from LAILA, containing the prompt (P7) with its English translation, essay (070773) without translation, to preserve Arabic-specific linguistic errors, and annotations (scores) with their English translation.
  • Figure 3: Overall score distributions per trait in LAILA.
  • Figure 4: Score distributions across prompts and traits in LAILA.
  • Figure 5: Distribution of essay lengths in LAILA.
  • ...and 1 more figures