Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Integrating AI in College Education: Positive yet Mixed Experiences with ChatGPT

Xinrui Song, Jiajin Zhang, Pingkun Yan, Juergen Hahn, Uwe Kruger, Hisham Mohamed, Ge Wang

TL;DR

The paper evaluates integrating AI chatbots in higher education through a semester-long trial in a medical imaging course, comparing baseline ChatGPT-3.5, ChatGPT-4 Turbo, and course-specific ChatGe-V1/V2 tools. It collects student surveys, Q&A evaluations, and exam grades to assess engagement, perceived value, and educational impact, while exploring how version, prompts, and instructor framing influence adoption. Findings indicate that ChatGPT improves information access and interactivity but raises accuracy concerns and underscores the need for explicit usage guidelines; tailored ChatGe-V2 can match or exceed performance of other options and even substitute some TA tasks. Promotion by instructors significantly boosts adoption, with AI-assisted answering offering substantial reductions in TA workload (instantaneous vs 15–20 minutes). The study informs practical guidelines for safe, effective AI-assisted learning in higher education and highlights opportunities and caveats for scalable AI-enabled pedagogy.

Abstract

The integration of artificial intelligence (AI) chatbots into higher education marks a shift towards a new generation of pedagogical tools, mirroring the arrival of milestones like the internet. With the launch of ChatGPT-4 Turbo in November 2023, we developed a ChatGPT-based teaching application (https://chat.openai.com/g/g-1imx1py4K-chatge-medical-imaging) and integrated it into our undergraduate medical imaging course in the Spring 2024 semester. This study investigates the use of ChatGPT throughout a semester-long trial, providing insights into students' engagement, perception, and the overall educational effectiveness of the technology. We systematically collected and analyzed data concerning students' interaction with ChatGPT, focusing on their attitudes, concerns, and usage patterns. The findings indicate that ChatGPT offers significant advantages such as improved information access and increased interactivity, but its adoption is accompanied by concerns about the accuracy of the information provided and the necessity for well-defined guidelines to optimize its use.

Integrating AI in College Education: Positive yet Mixed Experiences with ChatGPT

TL;DR

The paper evaluates integrating AI chatbots in higher education through a semester-long trial in a medical imaging course, comparing baseline ChatGPT-3.5, ChatGPT-4 Turbo, and course-specific ChatGe-V1/V2 tools. It collects student surveys, Q&A evaluations, and exam grades to assess engagement, perceived value, and educational impact, while exploring how version, prompts, and instructor framing influence adoption. Findings indicate that ChatGPT improves information access and interactivity but raises accuracy concerns and underscores the need for explicit usage guidelines; tailored ChatGe-V2 can match or exceed performance of other options and even substitute some TA tasks. Promotion by instructors significantly boosts adoption, with AI-assisted answering offering substantial reductions in TA workload (instantaneous vs 15–20 minutes). The study informs practical guidelines for safe, effective AI-assisted learning in higher education and highlights opportunities and caveats for scalable AI-enabled pedagogy.

Abstract

The integration of artificial intelligence (AI) chatbots into higher education marks a shift towards a new generation of pedagogical tools, mirroring the arrival of milestones like the internet. With the launch of ChatGPT-4 Turbo in November 2023, we developed a ChatGPT-based teaching application (https://chat.openai.com/g/g-1imx1py4K-chatge-medical-imaging) and integrated it into our undergraduate medical imaging course in the Spring 2024 semester. This study investigates the use of ChatGPT throughout a semester-long trial, providing insights into students' engagement, perception, and the overall educational effectiveness of the technology. We systematically collected and analyzed data concerning students' interaction with ChatGPT, focusing on their attitudes, concerns, and usage patterns. The findings indicate that ChatGPT offers significant advantages such as improved information access and increased interactivity, but its adoption is accompanied by concerns about the accuracy of the information provided and the necessity for well-defined guidelines to optimize its use.
Paper Structure (9 sections, 2 figures)

This paper contains 9 sections, 2 figures.

Table of Contents

  1. Introduction
  2. Methods
  3. Results and Discussions
  4. Students' Survey Results on ChatGPT Usage and Assessment
  5. Students' Preference over Different Versions of ChatGPT Tools
  6. Usage of ChatGPT Influenced by Promotions and Precaution
  7. ChatGPT for TA Workload Reduction
  8. Data and Code Availability Statement
  9. Competing Interest

Figures (2)

  • Figure 1: Visualization of the students' feedback on AI-assisted course learning. The sub-figures (a)-(e) reflect the students' ChatGPT usage and impression. The sub-figure (f depicts the students' evaluation on various aspects of the chatbots. (a) The percentage of the students using ChatGPT in the two classes; (b) the use of ChatGPT; (c) the number of hours of the ChatGPT usage per week; (d) the usage of different chatbot versions; (e) the best aspects of these chatbots; and (f) the students' answers to six questions (see the SI for details) on various aspects of the chatbots.
  • Figure 2: Students' course content Q&A preference. The bar plot statistically compares students' preference for the answers generated by either ChatGPT or TAs to typical course content-related questions. Q1 and Q2 were given after exam 1, and students responses were then collected to develop ChatGeV2.