Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Evidence-Decision-Feedback: Theory-Driven Adaptive Scaffolding for LLM Agents

Clayton Cohn, Siyuan Guo, Surya Rayala, Hanchen David Wang, Naveeduddin Mohammed, Umesh Timalsina, Shruti Jain, Angela Eeds, Menton Deweese, Pamela J. Osborn Popp, Rebekah Stanton, Shakeera Walker, Meiyi Ma, Gautam Biswas

TL;DR

This paper tackles the challenge of personalized, interpretable LLM-based tutoring that avoids overreliance on agents. It proposes the Evidence-Decision-Feedback (EDF) framework, grounding adaptive scaffolding in evidence-centered design, stealth assessment, and learning theories such as SCT and ZPD, and instantiates it with Copa, a four-subagent collaborative peer agent for STEM+C learning. In authentic high school classrooms (n=33 dyads), Copa demonstrates adaptive scaffolding that fades with mastery, aligns verbal explanations with task progress, and maintains interpretability through traceable evidentiary reasoning, while reducing learner dependence on the agent. The findings highlight both the potential and the challenges of deploying theory-driven, agentic LLM systems in real-world education, emphasizing the need for learner buy-in, further causal evaluation, and broader consideration of learner attributes for scalable impact.

Abstract

Multi-agent LLM architectures offer opportunities for pedagogical agents to help students construct domain knowledge and develop critical-thinking skills, yet many operate on a "one-size-fits-all" basis, limiting their ability to provide personalized support. To address this, we introduce Evidence-Decision-Feedback (EDF), a theoretical framework for adaptive scaffolding using LLMs. EDF integrates elements of intelligent tutoring systems and agentic behavior by organizing interactions around evidentiary inference, pedagogical decision-making, and adaptive feedback. We instantiate EDF through Copa, an agentic collaborative peer agent for STEM+C problem-solving. In an authentic high school classroom study, we show that EDF-aligned interactions align feedback with students' demonstrated understanding and task mastery; promote gradual scaffold fading; and support interpretable, evidence-grounded explanations without fostering overreliance.

Evidence-Decision-Feedback: Theory-Driven Adaptive Scaffolding for LLM Agents

TL;DR

This paper tackles the challenge of personalized, interpretable LLM-based tutoring that avoids overreliance on agents. It proposes the Evidence-Decision-Feedback (EDF) framework, grounding adaptive scaffolding in evidence-centered design, stealth assessment, and learning theories such as SCT and ZPD, and instantiates it with Copa, a four-subagent collaborative peer agent for STEM+C learning. In authentic high school classrooms (n=33 dyads), Copa demonstrates adaptive scaffolding that fades with mastery, aligns verbal explanations with task progress, and maintains interpretability through traceable evidentiary reasoning, while reducing learner dependence on the agent. The findings highlight both the potential and the challenges of deploying theory-driven, agentic LLM systems in real-world education, emphasizing the need for learner buy-in, further causal evaluation, and broader consideration of learner attributes for scalable impact.

Abstract

Multi-agent LLM architectures offer opportunities for pedagogical agents to help students construct domain knowledge and develop critical-thinking skills, yet many operate on a "one-size-fits-all" basis, limiting their ability to provide personalized support. To address this, we introduce Evidence-Decision-Feedback (EDF), a theoretical framework for adaptive scaffolding using LLMs. EDF integrates elements of intelligent tutoring systems and agentic behavior by organizing interactions around evidentiary inference, pedagogical decision-making, and adaptive feedback. We instantiate EDF through Copa, an agentic collaborative peer agent for STEM+C problem-solving. In an authentic high school classroom study, we show that EDF-aligned interactions align feedback with students' demonstrated understanding and task mastery; promote gradual scaffold fading; and support interpretable, evidence-grounded explanations without fostering overreliance.
Paper Structure (13 sections, 1 equation, 4 figures, 2 tables)

This paper contains 13 sections, 1 equation, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Theoretical Framework
  4. Copa
  5. Evaluation
  6. RQ1: Scaffolding Adaptivity
  7. RQ2: Understanding-Mastery Alignment
  8. RQ3: Student Reliance on Agent
  9. RQ4: Feedback Interpretability
  10. Student and Teacher Perceptions
  11. Discussion and Conclusions
  12. Acknowledgments.
  13. Disclosure of Interests.

Figures (4)

  • Figure 1: Evidence-Decision-Feedback (EDF) framework.
  • Figure 2: Copa, pictured within the C2STEM learning environment.
  • Figure 3: Copa and its four sub-agents. Colors correspond to the EDF modules presented in Figure \ref{['fig:edf']}. Gray boxes = data stores. Thick edges = two-way relationships.
  • Figure 4: (a) Correlation between probing success rate and mastery deciles (RQ2). (b) Correlation between student requests for agent support and mastery deciles (RQ3).