Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Insights from the Frontline: GenAI Utilization Among Software Engineering Students

Rudrajit Choudhuri, Ambareesh Ramakrishnan, Amreeta Chatterjee, Bianca Trinkenreich, Igor Steinmacher, Marco Gerosa, Anita Sarma

TL;DR

The paper addresses how generative AI tools influence software engineering students' learning and project work, aiming to guide educators on when and how to integrate genAI effectively. It adopts a qualitative approach with 16 reflective interviews, validated by instructors, and analyzes data via reflexive thematic analysis to reveal when/how genAI is used and why challenges arise. Key findings show that genAI aids incremental learning (L2) and initial implementations (I1) but can hinder novices (L1) and advanced work (I2) due to intrinsic AI faults and knowledge gaps, which impact learning, task progress, self-efficacy, and adoption. The study translates these insights into practical recommendations for AI literacy, prompt engineering, and curriculum design, encouraging thoughtful, reflective use of genAI rather than punitive policies and underscoring the need for scaffolding genAI in SE education.

Abstract

Generative AI (genAI) tools (e.g., ChatGPT, Copilot) have become ubiquitous in software engineering (SE). As SE educators, it behooves us to understand the consequences of genAI usage among SE students and to create a holistic view of where these tools can be successfully used. Through 16 reflective interviews with SE students, we explored their academic experiences of using genAI tools to complement SE learning and implementations. We uncover the contexts where these tools are helpful and where they pose challenges, along with examining why these challenges arise and how they impact students. We validated our findings through member checking and triangulation with instructors. Our findings provide practical considerations of where and why genAI should (not) be used in the context of supporting SE students.

Insights from the Frontline: GenAI Utilization Among Software Engineering Students

TL;DR

The paper addresses how generative AI tools influence software engineering students' learning and project work, aiming to guide educators on when and how to integrate genAI effectively. It adopts a qualitative approach with 16 reflective interviews, validated by instructors, and analyzes data via reflexive thematic analysis to reveal when/how genAI is used and why challenges arise. Key findings show that genAI aids incremental learning (L2) and initial implementations (I1) but can hinder novices (L1) and advanced work (I2) due to intrinsic AI faults and knowledge gaps, which impact learning, task progress, self-efficacy, and adoption. The study translates these insights into practical recommendations for AI literacy, prompt engineering, and curriculum design, encouraging thoughtful, reflective use of genAI rather than punitive policies and underscoring the need for scaffolding genAI in SE education.

Abstract

Generative AI (genAI) tools (e.g., ChatGPT, Copilot) have become ubiquitous in software engineering (SE). As SE educators, it behooves us to understand the consequences of genAI usage among SE students and to create a holistic view of where these tools can be successfully used. Through 16 reflective interviews with SE students, we explored their academic experiences of using genAI tools to complement SE learning and implementations. We uncover the contexts where these tools are helpful and where they pose challenges, along with examining why these challenges arise and how they impact students. We validated our findings through member checking and triangulation with instructors. Our findings provide practical considerations of where and why genAI should (not) be used in the context of supporting SE students.

Paper Structure

This paper contains 15 sections, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Interview Planning
  5. Data Collection and Analysis
  6. Generative AI Usage Among SE Students (RQ1)
  7. When do SE students use genAI?
  8. How do they use it?
  9. Perceived benefits and challenges
  10. Antecedents & Impacts of Challenges (RQ2)
  11. Causes
  12. Consequences
  13. Discussion
  14. Threats to Validity
  15. Conclusion

Figures (2)

  • Figure 1: Perceived benefits and challenges of genAI usage among students: Benefits were in incremental learning (L2) & initial implementations (I1), with challenges in initial learning (L1) & advanced implementations (I2). Stars indicate perceptions unique to the quadrant.
  • Figure 2: Associations between genAI's intrinsic issues (faults and gaps), challenges (C1-C5), and the resulting impacts. Issues in genAI contributed to various challenges encountered by participants, which subsequently had impacts on learning, task, self, and genAI adoption.