Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AI Hallucination from Students' Perspective: A Thematic Analysis

Abdulhadi Shoufan, Ahmad-Azmi-Abdelhamid Esmaeil

TL;DR

Vulnerabilities in AI-supported learning are illuminated and highlight the need for explicit instruction in verification protocols, accurate mental models of generative AI, and awareness of behaviors such as sycophancy and confident delivery that obscure inaccuracy.

Abstract

As students increasingly rely on large language models, hallucinations pose a growing threat to learning. To mitigate this, AI literacy must expand beyond prompt engineering to address how students should detect and respond to LLM hallucinations. To support this, we need to understand how students experience hallucinations, how they detect them, and why they believe they occur. To investigate these questions, we asked university students three open-ended questions about their experiences with AI hallucinations, their detection strategies, and their mental models of why hallucinations occur. Sixty-three students responded to the survey. Thematic analysis of their responses revealed that reported hallucination issues primarily relate to incorrect or fabricated citations, false information, overconfident but misleading responses, poor adherence to prompts, persistence in incorrect answers, and sycophancy. To detect hallucinations, students rely either on intuitive judgment or on active verification strategies, such as cross-checking with external sources or re-prompting the model. Students' explanations for why hallucinations occur reflected several mental models, including notable misconceptions. Many described AI as a research engine that fabricates information when it cannot locate an answer in its "database." Others attributed hallucinations to issues with training data, inadequate prompting, or the model's inability to understand or verify information. These findings illuminate vulnerabilities in AI-supported learning and highlight the need for explicit instruction in verification protocols, accurate mental models of generative AI, and awareness of behaviors such as sycophancy and confident delivery that obscure inaccuracy. The study contributes empirical evidence for integrating hallucination awareness and mitigation into AI literacy curricula.

AI Hallucination from Students' Perspective: A Thematic Analysis

TL;DR

Vulnerabilities in AI-supported learning are illuminated and highlight the need for explicit instruction in verification protocols, accurate mental models of generative AI, and awareness of behaviors such as sycophancy and confident delivery that obscure inaccuracy.

Abstract

As students increasingly rely on large language models, hallucinations pose a growing threat to learning. To mitigate this, AI literacy must expand beyond prompt engineering to address how students should detect and respond to LLM hallucinations. To support this, we need to understand how students experience hallucinations, how they detect them, and why they believe they occur. To investigate these questions, we asked university students three open-ended questions about their experiences with AI hallucinations, their detection strategies, and their mental models of why hallucinations occur. Sixty-three students responded to the survey. Thematic analysis of their responses revealed that reported hallucination issues primarily relate to incorrect or fabricated citations, false information, overconfident but misleading responses, poor adherence to prompts, persistence in incorrect answers, and sycophancy. To detect hallucinations, students rely either on intuitive judgment or on active verification strategies, such as cross-checking with external sources or re-prompting the model. Students' explanations for why hallucinations occur reflected several mental models, including notable misconceptions. Many described AI as a research engine that fabricates information when it cannot locate an answer in its "database." Others attributed hallucinations to issues with training data, inadequate prompting, or the model's inability to understand or verify information. These findings illuminate vulnerabilities in AI-supported learning and highlight the need for explicit instruction in verification protocols, accurate mental models of generative AI, and awareness of behaviors such as sycophancy and confident delivery that obscure inaccuracy. The study contributes empirical evidence for integrating hallucination awareness and mitigation into AI literacy curricula.
Paper Structure (30 sections, 9 figures)

This paper contains 30 sections, 9 figures.

Table of Contents

  1. Introduction
  2. Methodology
  3. Study design
  4. Participants and context
  5. Open-ended questions
  6. Thematic analysis of students' responses
  7. Results
  8. RQ1 (What types of hallucinations do students report encountering?)
  9. Hallucination issues
  10. Hallucinaion domains
  11. RQ2 (What strategies do students use to identify hallucinations?)
  12. Students' perceptions of AI hallucination
  13. Detecting hallucination by verification
  14. RQ3 (How do students explain the phenomenon of LLM hallucination?)
  15. Hallucination due to text generation issues
  16. ...and 15 more sections

Figures (9)

  • Figure 1: A total of 152 students’ comments about their experiences with AI hallucination were coded into four main themes.
  • Figure 2: Students' comments about AI hallucination issues were grouped into seven subthemes.
  • Figure 3: Domains students mentioned when they reported their experience with AI hallucination.
  • Figure 4: Students identify AI hallucination either by their perception and direct judgment or by using some kind of verification.
  • Figure 5: Issues that raise students' perception of hallucination
  • ...and 4 more figures