Evaluation of Large Language Models' educational feedback in Higher Education: potential, limitations and implications for educational practice
Daniele Agostini, Federica Picasso
TL;DR
This paper investigates the potential of seven large language models to generate descriptive, rubric-based feedback in a higher-education setting, using Hughes, Smith, and Creese’s framework to assess constructiveness, structure, and formative value. Through a controlled design with 142 students across 35 groups, the study combines expert human evaluation with AI-based assessments to compare how different LLMs address praise, critique, advice, and clarification within a structured task redesign activity. The findings reveal that LLMs can produce well-formed, actionable feedback, though quality and emphasis vary by model, particularly in error correction and clarification, underscoring the importance of explicit prompting and rubric-driven evaluation. The work highlights the practical potential of AI-assisted feedback to reduce educator workload while emphasizing the need for careful model selection, prompt design, and ethical considerations to preserve fairness and educational quality in higher education.
Abstract
The importance of managing feedback practices in higher education has been widely recognised, as they play a crucial role in enhancing teaching, learning, and assessment processes. In today's educational landscape, feedback practices are increasingly influenced by technological advancements, particularly artificial intelligence (AI). Understanding the impact of AI on feedback generation is essential for identifying its potential benefits and establishing effective implementation strategies. This study examines how AI-generated feedback supports student learning using a well-established analytical framework. Specifically, feedback produced by different Large Language Models (LLMs) was assessed in relation to student-designed projects within a training course on inclusive teaching and learning. The evaluation process involved providing seven LLMs with a structured rubric, developed by the university instructor, which defined specific criteria and performance levels. The LLMs were tasked with generating both quantitative assessments and qualitative feedback based on this rubric. The AI-generated feedback was then analysed using Hughes, Smith, and Creese's framework to evaluate its structure and effectiveness in fostering formative learning experiences. Overall, these findings indicate that LLMs can generate well-structured feedback and hold great potential as a sustainable and meaningful feedback tool, provided they are guided by clear contextual information and a well-defined instructions that will be explored further in the conclusions.