Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Leveraging Peer, Self, and Teacher Assessments for Generative AI-Enhanced Feedback

Alvaro Becerra, Ruth Cobos

TL;DR

The paper tackles the challenge of delivering timely, high-quality feedback in large engineering courses by evaluating a GenAI-augmented feedback system (AICoFe) that combines teacher, peer, and self assessments of oral presentations. It provides a rigorous empirical analysis of agreement, correlation, and biases among evaluators using a validated rubric, and demonstrates the limitations of equal weighting in aggregating inputs. The authors propose a weighted input integration and bias-detection framework for the GenAI component (GePeTo), enabling context-aware, pedagogy-aligned feedback with human oversight. The findings and design recommendations offer a practical pathway to scalable, trustworthy AI-assisted feedback that preserves pedagogical validity and transparency in engineering education.

Abstract

Providing timely and meaningful feedback remains a persistent challenge in higher education, especially in large courses where teachers must balance formative depth with scalability. Recent advances in Generative Artificial Intelligence (GenAI) offer new opportunities to support feedback processes while maintaining human oversight. This paper presents an study conducted within the AICoFe (AI-based Collaborative Feedback) system, which integrates teacher, peer, and self-assessments of engineering students' oral presentations. Using a validated rubric, 46 evaluation sets were analyzed to examine agreement, correlation, and bias across evaluators. The analyses revealed consistent overall alignment among sources but also systematic variations in scoring behavior, reflecting distinct evaluative perspectives. These findings informed the proposal of an enhanced GenAI model within AICoFe system, designed to integrate human assessments through weighted input aggregation, bias detection, and context-aware feedback generation. The study contributes empirical evidence and design principles for developing GenAI-based feedback systems that combine data-based efficiency with pedagogical validity and transparency.

Leveraging Peer, Self, and Teacher Assessments for Generative AI-Enhanced Feedback

TL;DR

The paper tackles the challenge of delivering timely, high-quality feedback in large engineering courses by evaluating a GenAI-augmented feedback system (AICoFe) that combines teacher, peer, and self assessments of oral presentations. It provides a rigorous empirical analysis of agreement, correlation, and biases among evaluators using a validated rubric, and demonstrates the limitations of equal weighting in aggregating inputs. The authors propose a weighted input integration and bias-detection framework for the GenAI component (GePeTo), enabling context-aware, pedagogy-aligned feedback with human oversight. The findings and design recommendations offer a practical pathway to scalable, trustworthy AI-assisted feedback that preserves pedagogical validity and transparency in engineering education.

Abstract

Providing timely and meaningful feedback remains a persistent challenge in higher education, especially in large courses where teachers must balance formative depth with scalability. Recent advances in Generative Artificial Intelligence (GenAI) offer new opportunities to support feedback processes while maintaining human oversight. This paper presents an study conducted within the AICoFe (AI-based Collaborative Feedback) system, which integrates teacher, peer, and self-assessments of engineering students' oral presentations. Using a validated rubric, 46 evaluation sets were analyzed to examine agreement, correlation, and bias across evaluators. The analyses revealed consistent overall alignment among sources but also systematic variations in scoring behavior, reflecting distinct evaluative perspectives. These findings informed the proposal of an enhanced GenAI model within AICoFe system, designed to integrate human assessments through weighted input aggregation, bias detection, and context-aware feedback generation. The study contributes empirical evidence and design principles for developing GenAI-based feedback systems that combine data-based efficiency with pedagogical validity and transparency.

Paper Structure

This paper contains 21 sections, 4 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Rubrics in Peer, Self, and Teacher Assessment
  4. Comparing Peer, Self, and Teacher Assessment
  5. GenAI-Supported Feedback and Learning Analytics
  6. Methodology
  7. Context and Participants
  8. Assessment Instrument (Rubric)
  9. Data Collection Procedure
  10. Statistical Analyses
  11. Ethical Considerations
  12. Results
  13. Agreement Across Evaluators
  14. Correlation Across Evaluations
  15. Bias Analysis
  16. ...and 6 more sections

Figures (4)

  • Figure 1: Classroom setup of the peer assessment activity using the AICoFe system
  • Figure 2: Pairwise correlations between rubric-based and teacher's overall grades across evaluators (teacher, peers, and self-assessments). Each subplot displays the Pearson correlation coefficient ($r$) and the regression line (in red)
  • Figure 3: Distribution of scores across evaluation sources
  • Figure 4: Comparison between the Current Approach (top) and the Proposed Enhanced Approach (bottom) for Generative AI–based feedback within the AICoFe system