Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

How Do Teachers Create Pedagogical Chatbots?: Current Practices and Challenges

Minju Yoo, Hyoungwook Jin, Juho Kim

TL;DR

The paper investigates how K-12 teachers create LLM-based pedagogical chatbots and identifies practical practices and challenges across design, implementation, and deployment. Through semi-structured interviews with seven teachers, it reveals a preference for task-specific, lesson-aligned bots and highlights the need for clearer guidance and support structures. It contributes a design-space perspective, a proposal for interface-agent–assisted implementation, and a modular, reusable approach to chatbot interactions to reduce development burden and improve classroom integration. The findings advance teacher-centered AI tool design, offering actionable directions to empower educators to augment instruction with AI while maintaining instructional agency and student safety.

Abstract

AI chatbots have emerged as promising educational tools for personalized learning experiences, with advances in large language models (LLMs) enabling teachers to create and customize these chatbots for their specific classroom needs. However, there is a limited understanding of how teachers create pedagogical chatbots and integrate them into their lessons. Through semi-structured interviews with seven K-12 teachers, we examined their practices and challenges when designing, implementing, and deploying chatbots. Our findings revealed that teachers prioritize developing task-specific chatbots aligned with their lessons. Teachers engaged in various creation practices and had different challenges; novices in chatbot creation struggled mainly with initial design and technical implementation, while experienced teachers faced challenges with technical aspects and analyzing conversational data. Based on these insights, we explore approaches to supporting teachers' chatbot development and opportunities for designing future chatbot creation systems. This work provides foundational insights from teachers that can empower teacher-created chatbots, facilitating AI-augmented teaching.

How Do Teachers Create Pedagogical Chatbots?: Current Practices and Challenges

TL;DR

The paper investigates how K-12 teachers create LLM-based pedagogical chatbots and identifies practical practices and challenges across design, implementation, and deployment. Through semi-structured interviews with seven teachers, it reveals a preference for task-specific, lesson-aligned bots and highlights the need for clearer guidance and support structures. It contributes a design-space perspective, a proposal for interface-agent–assisted implementation, and a modular, reusable approach to chatbot interactions to reduce development burden and improve classroom integration. The findings advance teacher-centered AI tool design, offering actionable directions to empower educators to augment instruction with AI while maintaining instructional agency and student safety.

Abstract

AI chatbots have emerged as promising educational tools for personalized learning experiences, with advances in large language models (LLMs) enabling teachers to create and customize these chatbots for their specific classroom needs. However, there is a limited understanding of how teachers create pedagogical chatbots and integrate them into their lessons. Through semi-structured interviews with seven K-12 teachers, we examined their practices and challenges when designing, implementing, and deploying chatbots. Our findings revealed that teachers prioritize developing task-specific chatbots aligned with their lessons. Teachers engaged in various creation practices and had different challenges; novices in chatbot creation struggled mainly with initial design and technical implementation, while experienced teachers faced challenges with technical aspects and analyzing conversational data. Based on these insights, we explore approaches to supporting teachers' chatbot development and opportunities for designing future chatbot creation systems. This work provides foundational insights from teachers that can empower teacher-created chatbots, facilitating AI-augmented teaching.

Paper Structure

This paper contains 26 sections.

Table of Contents

  1. Introduction
  2. Teacher Interview
  3. Participants
  4. Interview Protocol
  5. Experienced Teachers
  6. Novice Teachers
  7. Teachers' Practices and Challenges
  8. Design: Outlining Lesson Plans and Chatbot Specifications
  9. [Practice] Teachers prefer to create chatbots by themselves to support their existing lesson plans.
  10. [Practice] Teachers intentionally bound chatbot capabilities to shape their pedagogical role.
  11. [Challenge] Lack of understanding of LLMs leads to over- or under-estimation of their roles in lessons.
  12. [Best Practice] Design references with detailed context can help teachers adapt chatbots.
  13. Implementation: Programming, Testing and Debugging Chatbots
  14. [Practice] Teachers leverage prompt templates for scaffolding and efficiency.
  15. [Practice] Teachers focus on testing chatbots for safety and pedagogical validity.
  16. ...and 11 more sections