Automated grading workflows for providing personalized feedback to open-ended data science assignments
Federica Zoe Ricci, Catalina Mari Medina, Mine Dogucu
TL;DR
The paper addresses the challenge of grading open-ended data science assignments, proposing an automated grading workflow and introducing the gradetools R package implemented in RStudio. The approach centers on rubric-driven, scalable feedback and GUI-based grading, with integrations to GitHub via ghclass to support reproducible teaching. Key contributions include detailing the three-phase grading workflow, rubric design and dynamic updates, on-the-fly unique feedback, progress logging, and support for diverse grading scenarios (projects, teams, GitHub-managed submissions). The work has practical impact by enabling efficient, fair, and reproducible feedback provision in data science education, while acknowledging adoption considerations and limitations. The authors also emphasize reproducibility and the growing relevance of automated tooling in the context of AI-assisted learning.
Abstract
Open-ended assignments - such as lab reports and semester-long projects - provide data science and statistics students with opportunities for developing communication, critical thinking, and creativity skills. However, providing grades and formative feedback to open-ended assignments can be very time consuming and difficult to do consistently across students. In this paper, we discuss the steps of a typical grading workflow and highlight which steps can be automated in an approach that we call automated grading workflow. We illustrate how gradetools, a new R package, implements this approach within RStudio to facilitate efficient and consistent grading while providing individualized feedback. By outlining the motivations behind the development of this package and the considerations underlying its design, we hope this article will provide data science and statistics educators with ideas for improving their grading workflows, possibly developing new grading tools or considering use gradetools as their grading workflow assistant.