Learning Agent-based Modeling with LLM Companions: Experiences of Novices and Experts Using ChatGPT & NetLogo Chat
John Chen, Xi Lu, Michael Rejtig, David Du, Ruth Bagley, Michael S. Horn, Uri J. Wilensky
TL;DR
This work presents NetLogo Chat, an LLM-assisted interface integrated with the NetLogo IDE to support learning and practicing agent-based modeling. An open-ended interview study with 30 participants reveals that experts perceive greater benefits and are more inclined to adopt LLMs, while novices face a knowledge gap that contributes to a larger behavioral gap. The authors propose design interventions around guidance, personalization, and integration, guided by constructionist learning theory, to bridge the novice-expert divide and improve learning outcomes. The findings offer actionable implications for designing LLM-based programming interfaces in computational modeling and ABM education, emphasizing iterative, source-backed workflows and deeper IDE integration.
Abstract
Large Language Models (LLMs) have the potential to fundamentally change the way people engage in computer programming. Agent-based modeling (ABM) has become ubiquitous in natural and social sciences and education, yet no prior studies have explored the potential of LLMs to assist it. We designed NetLogo Chat to support the learning and practice of NetLogo, a programming language for ABM. To understand how users perceive, use, and need LLM-based interfaces, we interviewed 30 participants from global academia, industry, and graduate schools. Experts reported more perceived benefits than novices and were more inclined to adopt LLMs in their workflow. We found significant differences between experts and novices in their perceptions, behaviors, and needs for human-AI collaboration. We surfaced a knowledge gap between experts and novices as a possible reason for the benefit gap. We identified guidance, personalization, and integration as major needs for LLM-based interfaces to support the programming of ABM.