Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

When Scaffolding Breaks: Investigating Student Interaction with LLM-Based Writing Support in Real-Time K-12 EFL Classrooms

Junho Myung, Hyunseung Lim, Hana Oh, Hyoungwook Jin, Nayeon Kang, So-Yeon Ahn, Hwajung Hong, Alice Oh, Juho Kim

TL;DR

<3-5 sentence high-level summary>

Abstract

Large language models (LLMs) are promising tools for scaffolding students' English writing skills, but their effectiveness in real-time K-12 classrooms remains underexplored. Addressing this gap, our study examines the benefits and limitations of using LLMs as real-time learning support, considering how classroom constraints, such as diverse proficiency levels and limited time, affect their effectiveness. We conducted a deployment study with 157 eighth-grade students in a South Korean middle school English class over six weeks. Our findings reveal that while scaffolding improved students' ability to compose grammatically correct sentences, this step-by-step approach demotivated lower-proficiency students and increased their system reliance. We also observed challenges to classroom dynamics, where extroverted students often dominated the teacher's attention, and the system's assistance made it difficult for teachers to identify struggling students. Based on these findings, we discuss design guidelines for integrating LLMs into real-time writing classes as inclusive educational tools.

When Scaffolding Breaks: Investigating Student Interaction with LLM-Based Writing Support in Real-Time K-12 EFL Classrooms

TL;DR

<3-5 sentence high-level summary>

Abstract

Large language models (LLMs) are promising tools for scaffolding students' English writing skills, but their effectiveness in real-time K-12 classrooms remains underexplored. Addressing this gap, our study examines the benefits and limitations of using LLMs as real-time learning support, considering how classroom constraints, such as diverse proficiency levels and limited time, affect their effectiveness. We conducted a deployment study with 157 eighth-grade students in a South Korean middle school English class over six weeks. Our findings reveal that while scaffolding improved students' ability to compose grammatically correct sentences, this step-by-step approach demotivated lower-proficiency students and increased their system reliance. We also observed challenges to classroom dynamics, where extroverted students often dominated the teacher's attention, and the system's assistance made it difficult for teachers to identify struggling students. Based on these findings, we discuss design guidelines for integrating LLMs into real-time writing classes as inclusive educational tools.

Paper Structure

This paper contains 39 sections, 6 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Challenges of Teaching Writing in K-12 EFL
  4. AI Tools for EFL Writing Support
  5. LLMs as Scaffolds in the Classroom
  6. System Design
  7. Interface Design
  8. Prompt Design
  9. Method
  10. Participants
  11. Study Setting
  12. Observation Study
  13. Written Assessment
  14. Analysis
  15. Result
  16. ...and 24 more sections

Figures (6)

  • Figure 1: The interface design of WriteAid. Students can view the lesson objectives in section (A), write their essays in section (B), and interact with the AI chatbot in section (C). From section (C), students can select features they want from the available options in the tab. All instructions are provided in Korean and are translated into English for this figure.
  • Figure 2: Distribution of the number of query-response pairs between students and the LLM during each semester.
  • Figure 3: Distribution of the question types asked across different student performance levels.
  • Figure 4: Example interaction patterns that emerged from the use of WriteAid. The interactions are drawn from actual data of students from high-, middle-, and low-performance groups. Text in pink was originally written in Korean and translated into English for reader's clarity.
  • Figure 5: The rubric used to grade students' final assessment. The essays were graded using an analytic rubric adapted from jacobs1981testing.
  • ...and 1 more figures