Implementation Considerations for Automated AI Grading of Student Work

TL;DR

This paper investigates the classroom implementation of AI-based grading in K-12 through a seven-week co-design study with 19 teachers. It triangulates platform logs, teacher surveys, and interviews to examine how AI-generated rubrics and feedback are used, trusted, and perceived by students. The findings show that AI provides rapid formative feedback and rubric support, but automated scores are inconsistently trusted, requiring teacher oversight and transparent interfaces. Students welcome fast feedback for revision but remain skeptical of AI-only grading, underscoring the need for human validation and relational accountability. The study offers design guidelines for trustworthy, teacher-centered AI assessment tools that preserve pedagogical agency while expanding feedback and revision opportunities in schools.

Abstract

This study explores the classroom implementation of an AI-powered grading platform in K-12 settings through a co-design pilot with 19 teachers. We combine platform usage logs, surveys, and qualitative interviews to examine how teachers use AI-generated rubrics and grading feedback. Findings reveal that while teachers valued the AI's rapid narrative feedback for formative purposes, they distrusted automated scoring and emphasized the need for human oversight. Students welcomed fast, revision-oriented feedback but remained skeptical of AI-only grading. We discuss implications for the design of trustworthy, teacher-centered AI assessment tools that enhance feedback while preserving pedagogical agency.

Figures (4)

• Figure 1:
• Figure 2:
• Figure 3: Teacher survey responses to the quality of the AI generated rubric and whether it was necessary to make changes.
• Figure 4: Teacher survey responses to the quality of the AI generated assessment feedback