Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Students' Perspective on AI Code Completion: Benefits and Challenges

Wannita Takerngsaksiri, Cleshan Warusavitarne, Christian Yaacoub, Matthew Hee Keng Hou, Chakkrit Tantithamthavorn

TL;DR

This paper investigates student perspectives on AI code completion in computer science education. It employs AutoAurora, an open-source VS Code extension built on StarCoder, and a two-session interview study with ten undergraduates, analyzed via grounded theory to produce a taxonomy of benefits, challenges, and expectations. The results indicate productivity gains and tutoring benefits but highlight concerns about over-reliance and assessment implications, with students calling for explainability and best-practices guidance. The work offers design and curriculum implications for education-focused AI coding assistants and contributes to understanding how to balance automation with authentic learning in programming education.

Abstract

AI Code Completion (e.g., GitHub's Copilot) has revolutionized how computer science students interact with programming languages. However, AI code completion has been studied from the developers' perspectives, not the students' perspectives who represent the future generation of our digital world. In this paper, we investigated the benefits, challenges, and expectations of AI code completion from students' perspectives. To facilitate the study, we first developed an open-source Visual Studio Code Extension tool AutoAurora, powered by a state-of-the-art large language model StarCoder, as an AI code completion research instrument. Next, we conduct an interview study with ten student participants and apply grounded theory to help analyze insightful findings regarding the benefits, challenges, and expectations of students on AI code completion. Our findings show that AI code completion enhanced students' productivity and efficiency by providing correct syntax suggestions, offering alternative solutions, and functioning as a coding tutor. However, the over-reliance on AI code completion may lead to a surface-level understanding of programming concepts, diminishing problem-solving skills and restricting creativity. In the future, AI code completion should be explainable and provide best coding practices to enhance the education process.

Students' Perspective on AI Code Completion: Benefits and Challenges

TL;DR

This paper investigates student perspectives on AI code completion in computer science education. It employs AutoAurora, an open-source VS Code extension built on StarCoder, and a two-session interview study with ten undergraduates, analyzed via grounded theory to produce a taxonomy of benefits, challenges, and expectations. The results indicate productivity gains and tutoring benefits but highlight concerns about over-reliance and assessment implications, with students calling for explainability and best-practices guidance. The work offers design and curriculum implications for education-focused AI coding assistants and contributes to understanding how to balance automation with authentic learning in programming education.

Abstract

AI Code Completion (e.g., GitHub's Copilot) has revolutionized how computer science students interact with programming languages. However, AI code completion has been studied from the developers' perspectives, not the students' perspectives who represent the future generation of our digital world. In this paper, we investigated the benefits, challenges, and expectations of AI code completion from students' perspectives. To facilitate the study, we first developed an open-source Visual Studio Code Extension tool AutoAurora, powered by a state-of-the-art large language model StarCoder, as an AI code completion research instrument. Next, we conduct an interview study with ten student participants and apply grounded theory to help analyze insightful findings regarding the benefits, challenges, and expectations of students on AI code completion. Our findings show that AI code completion enhanced students' productivity and efficiency by providing correct syntax suggestions, offering alternative solutions, and functioning as a coding tutor. However, the over-reliance on AI code completion may lead to a surface-level understanding of programming concepts, diminishing problem-solving skills and restricting creativity. In the future, AI code completion should be explainable and provide best coding practices to enhance the education process.
Paper Structure (15 sections, 3 figures, 1 table)

This paper contains 15 sections, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background and Related Works
  3. AI Code Completion
  4. AI Code Completion User Studies
  5. AI Code Completion for Programming Education
  6. User Study Methodology
  7. Goal and Research Questions
  8. AutoAurora: An Open-source AI Code Completion Tool.
  9. Data Collection: An Interview Study
  10. Data Analysis: Grounded Theory
  11. Research Findings
  12. RQ1. What are the students' benefits when adopting the AI code completion tool?
  13. RQ2. What are the students' challenges when adopting the AI code completion tool?
  14. RQ3. What are the students' expectations when adopting the AI code completion tool?
  15. Conclusion

Figures (3)

  • Figure 1: An example of read/write file function suggested by our AutoAurora code completion tool in the Visual Studio Code.
  • Figure 2: An Overview of our Research Methodology for Students' Perspectives on AI Code Completion.
  • Figure 3: A summary of the benefits, challenges, and expectations of students' perspective when adopting AI code completion. The y-axis presents categories and the x-axis presents the percentage of students mentioning the category during the interviews.