Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Relying on LLMs: Student Practices and Instructor Norms are Changing in Computer Science Education

Xinrui Lin, Heyan Huang, Shumin Shi, John Vines

TL;DR

This paper investigates how CS students and instructors engage with LLMs across five education scenarios, revealing seven intents and varying levels of conflict with instructor norms. Using two linked qualitative studies, the authors map these intents to pragmatic practices, showing a shift from strict prohibition to process-based acceptance and the integration of usage records into assessment. They identify high-conflict intents that require guardrails and low-conflict intents where comprehension checks and metacognitive prompts support learning. The findings offer design recommendations for LLMs that balance efficiency with deep learning, guiding educators and AI designers toward scenario-aware, ethically grounded LLM integration in CS education.

Abstract

Prior research has raised concerns about students' over-reliance on large language models (LLMs) in higher education. This paper examines how Computer Science students and instructors engage with LLMs across five scenarios: "Writing", "Quiz", "Programming", "Project-based learning", and "Information retrieval". Through user studies with 16 students and 6 instructors, we identify 7 key intents, including increasingly complex student practices. Findings reveal varying levels of conflict between student practices and instructor norms, ranging from clear conflict in "Writing-generation" and "(Programming) quiz-solving", through partial conflict in "Programming project-implementation" and "Project-based learning", to broad agreement in "Writing-revision & ideation", "(Programming) quiz-correction" and "Info-query & summary". We document instructors are shifting from prohibiting to recognizing students' use of LLMs for high-quality work, integrating usage records into assessment grading. Finally, we propose LLM design guidelines: deploying default guardrails with game-like and empathetic interaction to prevent students from "deserting" LLMs, especially for "Writing-generation", while utilizing comprehension checks in low-conflict intents to promote learning.

Relying on LLMs: Student Practices and Instructor Norms are Changing in Computer Science Education

TL;DR

This paper investigates how CS students and instructors engage with LLMs across five education scenarios, revealing seven intents and varying levels of conflict with instructor norms. Using two linked qualitative studies, the authors map these intents to pragmatic practices, showing a shift from strict prohibition to process-based acceptance and the integration of usage records into assessment. They identify high-conflict intents that require guardrails and low-conflict intents where comprehension checks and metacognitive prompts support learning. The findings offer design recommendations for LLMs that balance efficiency with deep learning, guiding educators and AI designers toward scenario-aware, ethically grounded LLM integration in CS education.

Abstract

Prior research has raised concerns about students' over-reliance on large language models (LLMs) in higher education. This paper examines how Computer Science students and instructors engage with LLMs across five scenarios: "Writing", "Quiz", "Programming", "Project-based learning", and "Information retrieval". Through user studies with 16 students and 6 instructors, we identify 7 key intents, including increasingly complex student practices. Findings reveal varying levels of conflict between student practices and instructor norms, ranging from clear conflict in "Writing-generation" and "(Programming) quiz-solving", through partial conflict in "Programming project-implementation" and "Project-based learning", to broad agreement in "Writing-revision & ideation", "(Programming) quiz-correction" and "Info-query & summary". We document instructors are shifting from prohibiting to recognizing students' use of LLMs for high-quality work, integrating usage records into assessment grading. Finally, we propose LLM design guidelines: deploying default guardrails with game-like and empathetic interaction to prevent students from "deserting" LLMs, especially for "Writing-generation", while utilizing comprehension checks in low-conflict intents to promote learning.
Paper Structure (48 sections, 1 figure, 2 tables)

This paper contains 48 sections, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Student practices with LLMs
  4. Instructor views on LLMs for studying
  5. Interaction designs for learning with LLMs
  6. Research approach
  7. Method - Students
  8. Participants
  9. User study procedure
  10. Chat logs analysis
  11. Ethical discussion
  12. Data analysis
  13. Findings: Student intents and practices
  14. Writing
  15. Quiz
  16. ...and 33 more sections

Figures (1)

  • Figure 1: Scenarios, Student intents & practices, and Instructor norms (by conflict level)