Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Directive, Metacognitive or a Blend of Both? A Comparison of AI-Generated Feedback Types on Student Engagement, Confidence, and Outcomes

Omar Alsaiari, Nilufar Baghaei, Jason M. Lodge, Omid Noroozi, Dragan Gašević, Marie Boden, Hassan Khosravi

TL;DR

This study directly compares directive, metacognitive, and hybrid AI-generated feedback in a semester-long, large-scale randomized trial (N=329) within a university design/programming course using the RiPPLE platform. It shows that the feedback types are linguistically distinct (directive with imperatives, metacognitive with reflective prompts, hybrid balanced) and that while engagement time and overall outcomes are similar across conditions, revision behavior differs: metacognitive prompts yield fewer revisions, whereas hybrid feedback prompts the most revisions. Confidence remains high across all groups, and resource quality shows no significant differences, indicating that AI feedback can support learning without sacrificing perceived quality. The findings underscore the potential of hybrid AI feedback to balance immediate guidance with reflective practice, informing scalable design and implementation of AI-assisted feedback in higher education.

Abstract

Feedback is one of the most powerful influences on student learning, with extensive research examining how best to implement it in educational settings. Increasingly, feedback is being generated by artificial intelligence (AI), offering scalable and adaptive responses. Two widely studied approaches are directive feedback, which gives explicit explanations and reduces cognitive load to speed up learning, and metacognitive feedback which prompts learners to reflect, track their progress, and develop self-regulated learning (SRL) skills. While both approaches have clear theoretical advantages, their comparative effects on engagement, confidence, and quality of work remain underexplored. This study presents a semester-long randomised controlled trial with 329 students in an introductory design and programming course using an adaptive educational platform. Participants were assigned to receive directive, metacognitive, or hybrid AI-generated feedback that blended elements of both directive and metacognitive feedback. Results showed that revision behaviour differed across feedback conditions, with Hybrid prompting the most revisions compared to Directive and Metacognitive. Confidence ratings were uniformly high, and resource quality outcomes were comparable across conditions. These findings highlight the promise of AI in delivering feedback that balances clarity with reflection. Hybrid approaches, in particular, show potential to combine actionable guidance for immediate improvement with opportunities for self-reflection and metacognitive growth.

Directive, Metacognitive or a Blend of Both? A Comparison of AI-Generated Feedback Types on Student Engagement, Confidence, and Outcomes

TL;DR

This study directly compares directive, metacognitive, and hybrid AI-generated feedback in a semester-long, large-scale randomized trial (N=329) within a university design/programming course using the RiPPLE platform. It shows that the feedback types are linguistically distinct (directive with imperatives, metacognitive with reflective prompts, hybrid balanced) and that while engagement time and overall outcomes are similar across conditions, revision behavior differs: metacognitive prompts yield fewer revisions, whereas hybrid feedback prompts the most revisions. Confidence remains high across all groups, and resource quality shows no significant differences, indicating that AI feedback can support learning without sacrificing perceived quality. The findings underscore the potential of hybrid AI feedback to balance immediate guidance with reflective practice, informing scalable design and implementation of AI-assisted feedback in higher education.

Abstract

Feedback is one of the most powerful influences on student learning, with extensive research examining how best to implement it in educational settings. Increasingly, feedback is being generated by artificial intelligence (AI), offering scalable and adaptive responses. Two widely studied approaches are directive feedback, which gives explicit explanations and reduces cognitive load to speed up learning, and metacognitive feedback which prompts learners to reflect, track their progress, and develop self-regulated learning (SRL) skills. While both approaches have clear theoretical advantages, their comparative effects on engagement, confidence, and quality of work remain underexplored. This study presents a semester-long randomised controlled trial with 329 students in an introductory design and programming course using an adaptive educational platform. Participants were assigned to receive directive, metacognitive, or hybrid AI-generated feedback that blended elements of both directive and metacognitive feedback. Results showed that revision behaviour differed across feedback conditions, with Hybrid prompting the most revisions compared to Directive and Metacognitive. Confidence ratings were uniformly high, and resource quality outcomes were comparable across conditions. These findings highlight the promise of AI in delivering feedback that balances clarity with reflection. Hybrid approaches, in particular, show potential to combine actionable guidance for immediate improvement with opportunities for self-reflection and metacognitive growth.
Paper Structure (24 sections, 6 figures, 3 tables)

This paper contains 24 sections, 6 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methods
  3. Research Tool: The RiPPLE System
  4. Conditions
  5. Participants and Learning Context
  6. Study Design
  7. Data Analysis
  8. RQ1. Linguistic and structural features
  9. RQ2. Engagement with feedback
  10. Engagement time.
  11. Action on feedback.
  12. Event-flow transitions.
  13. Results
  14. RQ1. Linguistic and Structural Features
  15. RQ2: Engagement and Action on AI-generated feedback
  16. ...and 9 more sections

Figures (6)

  • Figure 1: Student resource creation process in RiPPLE
  • Figure 2: Creation stage on the RiPPLE platform showing the example MCQ used across all feedback conditions.
  • Figure 3: Comparison of directive, metacognitive, and hybrid AI-generated feedback types
  • Figure 4: Distribution of linguistic features across feedback conditions.
  • Figure 5: Task engagement flows across the three conditions (Directive, Metacognitive, Hybrid) using FOMM
  • ...and 1 more figures