Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

EduThink4AI: Bridging Educational Critical Thinking and Multi-Agent LLM Systems

Xinmeng Hou, Ziting Chang, Zhouquan Lu, Chen Wenli, Liang Wan, Wei Feng, Hai Hu, Qing Guo

TL;DR

This paper addresses the brittleness of current LLM educational tutors in promoting genuine critical thinking and resilience to adversarial prompts. It introduces EDU-Prompting, a modular, four-agent framework that bridges educational critical-thinking theories with LLM agent design through two phases: Phase I generates Raw Answers and Validity Cues via zero-shot agents, and Phase II applies Critique and Aggregation agents using zero-shot Chain-of-Thought reasoning to produce a final, bias-aware explanation. The approach is validated across standard benchmark suites for truthfulness, logical coherence, and bias, and is demonstrated in a full-stack college-writing application with three experimental conditions. Results show significant improvements in content truthfulness, logical soundness, and safety, while maintaining compatibility with existing prompting frameworks and enabling flexible deployment in educational settings.

Abstract

Large language models (LLMs) have demonstrated significant potential as educational tutoring agents, capable of tailoring hints, orchestrating lessons, and grading with near-human finesse across various academic domains. However, current LLM-based educational systems exhibit critical limitations in promoting genuine critical thinking, failing on over one-third of multi-hop questions with counterfactual premises, and remaining vulnerable to adversarial prompts that trigger biased or factually incorrect responses. To address these gaps, we propose \textbf{EDU-Prompting}, a novel multi-agent framework that bridges established educational critical thinking theories with LLM agent design to generate critical, bias-aware explanations while fostering diverse perspectives. Our systematic evaluation across theoretical benchmarks and practical college-level critical writing scenarios demonstrates that EDU-Prompting significantly enhances both content truthfulness and logical soundness in AI-generated educational responses. The framework's modular design enables seamless integration into existing prompting frameworks and educational applications, allowing practitioners to directly incorporate critical thinking catalysts that promote analytical reasoning and introduce multiple perspectives without requiring extensive system modifications.

EduThink4AI: Bridging Educational Critical Thinking and Multi-Agent LLM Systems

TL;DR

This paper addresses the brittleness of current LLM educational tutors in promoting genuine critical thinking and resilience to adversarial prompts. It introduces EDU-Prompting, a modular, four-agent framework that bridges educational critical-thinking theories with LLM agent design through two phases: Phase I generates Raw Answers and Validity Cues via zero-shot agents, and Phase II applies Critique and Aggregation agents using zero-shot Chain-of-Thought reasoning to produce a final, bias-aware explanation. The approach is validated across standard benchmark suites for truthfulness, logical coherence, and bias, and is demonstrated in a full-stack college-writing application with three experimental conditions. Results show significant improvements in content truthfulness, logical soundness, and safety, while maintaining compatibility with existing prompting frameworks and enabling flexible deployment in educational settings.

Abstract

Large language models (LLMs) have demonstrated significant potential as educational tutoring agents, capable of tailoring hints, orchestrating lessons, and grading with near-human finesse across various academic domains. However, current LLM-based educational systems exhibit critical limitations in promoting genuine critical thinking, failing on over one-third of multi-hop questions with counterfactual premises, and remaining vulnerable to adversarial prompts that trigger biased or factually incorrect responses. To address these gaps, we propose \textbf{EDU-Prompting}, a novel multi-agent framework that bridges established educational critical thinking theories with LLM agent design to generate critical, bias-aware explanations while fostering diverse perspectives. Our systematic evaluation across theoretical benchmarks and practical college-level critical writing scenarios demonstrates that EDU-Prompting significantly enhances both content truthfulness and logical soundness in AI-generated educational responses. The framework's modular design enables seamless integration into existing prompting frameworks and educational applications, allowing practitioners to directly incorporate critical thinking catalysts that promote analytical reasoning and introduce multiple perspectives without requiring extensive system modifications.

Paper Structure

This paper contains 21 sections, 5 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Methodology
  4. Phase I: Zero-shot Agents
  5. Agent I.
  6. Agent II.
  7. Phase II: Zero-Shot Chain-of-Thought Agents
  8. Agent III: Critique Agent
  9. Agent IV: Aggregation Agent
  10. Experiment
  11. Setups
  12. Datasets.
  13. Evaluation.
  14. User Study Application.
  15. Results on RQ1
  16. ...and 6 more sections

Figures (3)

  • Figure 1: Illustration of critical thinking in questioning and refining answers. Critical thinking about the question assesses its validity, while answer evaluation considers both content truthfulness and logical soundness.
  • Figure 2: EDU-Prompting Framework. There are four agents in charge of different matters: (I) brainstorming for context and key details, (II) a validity checker to explore whether a definitive answer exists, (III) a critique that formulates arguments and counterpoints, and (IV) a meta-review synthesizing findings into a conclusion.
  • Figure 3: (a) Normalized similarity between agent outputs and correct answers across pipeline stages (mean ± SD). (b) Cosine similarity between consecutive agents measuring inter-stage information transfer. Results show stable alignment throughout the pipeline, with dataset-specific variation reflecting differing task complexities. The Critique→Meta transition exhibits consistently high semantic continuity.