Bridging the AI Adoption Gap: Designing an Interactive Pedagogical Agent for Higher Education Instructors
Si Chen, Reid Metoyer, Khiem Le, Adam Acunin, Izzy Molnar, Alex Ambrose, James Lang, Nitesh Chawla, Ronald Metoyer
TL;DR
The paper investigates how to bridge the AI adoption gap among higher-education instructors by designing interactive pedagogical agents powered by LLMs. It employs a two-phase, human-centered methodology beginning with formative interviews with pedagogy experts and followed by a participatory design phase using two storyboard-driven interaction designs for AI-novice and AI-conservative instructors; suggestions are grounded via a Retrieval-Augmented Generation pipeline leveraging the ABLConnect database. The findings highlight design principles such as social transparency, incremental information collection, and peer validation, along with a grounded evaluation showing the prototype yields higher-quality, more actionable, and more reliable teaching suggestions than generic ChatGPT outputs. The work emphasizes the need for careful data privacy handling and iterative human-in-the-loop validation to prevent widening pedagogical divides, and it outlines concrete future directions like instructor simulators and expanded pedagogy resources to improve generalizability and equity in AI-assisted instruction.
Abstract
Instructors play a pivotal role in integrating AI into education, yet their adoption of AI-powered tools remains inconsistent. Despite this, limited research explores how to design AI tools that support broader instructor adoption. This study applies a human-centered design approach, incorporating qualitative methods, to investigate the design of interactive pedagogical agents that provide instructional suggestions in response to instructors' questions. We conducted a formative study involving interviews with five pedagogy experts to examine existing strategies for supporting instructors' pedagogical needs. Building on these insights, we facilitated a participatory design session with ten pedagogy experts, where participants reviewed a storyboard depicting a chatbot designed for instructors with varying levels of AI literacy and differing attitudes toward AI. Experts also evaluated the quality of LLM-generated suggestions based on common teaching challenges. Our findings highlight the need for chatbot interactions that foster trust, especially for AI-conservative instructors. Experts emphasized the importance of social transparency (for example, showing how peers use the tool) and allowing instructors to flexibly control how much or how little they engage with the system. We also propose design recommendations to enhance the quality of AI-generated teaching suggestions, such as adapting them to reflect instructors' prior teaching experience. This work underscores the urgent need to support AI-conservative instructors, as AI literacy and attitudes are closely intertwined. Without thoughtful design, there is a risk of widening pedagogical divides and reducing students' learning opportunities.