Qayyem: A Real-time Platform for Scoring Proficiency of Arabic Essays

TL;DR

Qayyem is a Web-based platform designed to support Arabic AES by providing an integrated workflow for assignment creation, batch essay upload, scoring configuration, and per-trait essay evaluation, allowing instructors to access advanced scoring services through a user-friendly interface.

Abstract

Over the past years, Automated Essay Scoring (AES) systems have gained increasing attention as scalable and consistent solutions for assessing the proficiency of student writing. Despite recent progress, support for Arabic AES remains limited due to linguistic complexity and scarcity of large publicly-available annotated datasets. In this work, we present Qayyem, a Web-based platform designed to support Arabic AES by providing an integrated workflow for assignment creation, batch essay upload, scoring configuration, and per-trait essay evaluation. Qayyem abstracts the technical complexity of interacting with scoring server APIs, allowing instructors to access advanced scoring services through a user-friendly interface. The platform deploys a number of state-of-the-art Arabic essay scoring models with different effectiveness and efficiency figures.

Figures (6)

• Figure 1: Qayyem workflow: (1) Prompt and Rubric Design, where the writing prompt and scoring rubric are defined; (2) Assignment Setup, where scoring traits and evaluation settings are configured; (3) Scoring, where essays are auto-evaluated across selected traits; and (4) Review, where results are inspected and optionally refined.
• Figure 2: The assignment setup interface in Arabic, with English header translations. English translations of the prompt and rubric are provided in Appendix \ref{['rubric-examples']}.
• Figure 3: Qayyem's High-level Architecture Diagram
• Figure 4: Scoring effectiveness (in QWK) vs. per-essay inference time (in msec) across deployed models.
• Figure 5: Example of a writing prompt from LAILA.