Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Strategies for Creating Uncertainty in the AI Era to Trigger Students Critical Thinking: Pedagogical Design, Assessment Rubric, and Exam System

Ahmad Samer Wazan

TL;DR

This paper addresses how AI challenges traditional assessments by enabling easy generation of correct answers without true understanding. It proposes an uncertainty-based pedagogy that leverages AI limitations and Socratic dialogue to stimulate critical thinking, paired with MindMosaicAIExam, a configurable exam system that records reasoning traces and controls AI behavior per question. A process-oriented evaluation rubric, inspired by the Delphi framework and adapted French critical-thinking frameworks, assesses the development of reasoning rather than merely final answers. The approach aims to preserve cognitive effort and provide actionable tools for instructors to design AI-assisted courses and scalable analytics.

Abstract

Generative AI challenges traditional assessments by allowing students to produce correct answers without demonstrating understanding or reasoning. Rather than prohibiting AI, this work argues that one way to integrate AI into education is by creating uncertain situations with the help of AI models and using thinking-oriented teaching approaches, where uncertainty is a central pedagogical concept for stimulating students critical thinking. Drawing on epistemology and critical thinking research studies, we propose designing learning activities and assessments around the inherent limitations of both AI models and instructors. This encourages students to reason, question, and justify their final answers. We show how explicitly controlling AI behavior during exams (such as preventing direct answers or generating plausible but flawed responses) prevents AI from becoming a shortcut to certainty. To support this pedagogy, we introduce MindMosaicAIExam, an exam system that integrates controllable AI tools and requires students to provide initial answers, critically evaluate AI outputs, and iteratively refine their reasoning. We also present an evaluation rubric designed to assess critical thinking based on students reasoning artifacts collected by the exam system.

Strategies for Creating Uncertainty in the AI Era to Trigger Students Critical Thinking: Pedagogical Design, Assessment Rubric, and Exam System

TL;DR

This paper addresses how AI challenges traditional assessments by enabling easy generation of correct answers without true understanding. It proposes an uncertainty-based pedagogy that leverages AI limitations and Socratic dialogue to stimulate critical thinking, paired with MindMosaicAIExam, a configurable exam system that records reasoning traces and controls AI behavior per question. A process-oriented evaluation rubric, inspired by the Delphi framework and adapted French critical-thinking frameworks, assesses the development of reasoning rather than merely final answers. The approach aims to preserve cognitive effort and provide actionable tools for instructors to design AI-assisted courses and scalable analytics.

Abstract

Generative AI challenges traditional assessments by allowing students to produce correct answers without demonstrating understanding or reasoning. Rather than prohibiting AI, this work argues that one way to integrate AI into education is by creating uncertain situations with the help of AI models and using thinking-oriented teaching approaches, where uncertainty is a central pedagogical concept for stimulating students critical thinking. Drawing on epistemology and critical thinking research studies, we propose designing learning activities and assessments around the inherent limitations of both AI models and instructors. This encourages students to reason, question, and justify their final answers. We show how explicitly controlling AI behavior during exams (such as preventing direct answers or generating plausible but flawed responses) prevents AI from becoming a shortcut to certainty. To support this pedagogy, we introduce MindMosaicAIExam, an exam system that integrates controllable AI tools and requires students to provide initial answers, critically evaluate AI outputs, and iteratively refine their reasoning. We also present an evaluation rubric designed to assess critical thinking based on students reasoning artifacts collected by the exam system.
Paper Structure (8 sections, 10 figures, 7 tables)

This paper contains 8 sections, 10 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Education Challenges Caused by AI
  3. Education and Critical Thinking
  4. Can Critical Thinking Be Universally Defined and Evaluated?
  5. Our Pedagogical Approach to Enforce Critical Thinking
  6. Evaluation Rubric for Our Students' Critical Thinking
  7. MindMosaicAIExam System
  8. Limitations, Conclusions and Future Works

Figures (10)

  • Figure 1: Simplified epistemological process: how beliefs become knowledge through critical examination
  • Figure 2: AI and Memorization have the same problem: bypassing human understanding (Figure adapted with minor modifications from kapoor2025could)
  • Figure 3: French Critical Thinking Evaluation Framework
  • Figure 4: Understand, Check and Ask Follow-up Questions Loop
  • Figure 5: Process- and result-oriented rubric for measuring critical thinking in AI-allowed exams.
  • ...and 5 more figures