Who's the Leader? Analyzing Novice Workflows in LLM-Assisted Debugging of Machine Learning Code

TL;DR

This study investigates how novices interact with LLMs during machine learning debugging, a domain characterized by high complexity and low verifiability. Through a formative study with eight participants using ChatGPT to fix a buggy Random Forest script on the UCI Adult dataset, the authors identify two reliance styles—leading and being led-by—and link these to task performance and learning outcomes, including a strong correlation between holdout F1 and initial ML knowledge ($r=0.93$, $p<0.001$). The results reveal that leading behavior is associated with better performance, while led-by and open-ended prompts can induce cognitive overload or over-/under-reliance, highlighting risks to cognitive engagement and downstream learning. The discussion proposes LLM- and user-side augmentations, plus bidirectional strategies like mutual theory of mind, to improve novice-LLM collaboration and promote accurate mental models. Overall, the work sheds light on how to design novice-oriented LLM-assisted workflows that balance immediate task success with meaningful long-term learning.

Abstract

While LLMs are often touted as tools for democratizing specialized knowledge to beginners, their actual effectiveness for improving task performance and learning is still an open question. It is known that novices engage with LLMs differently from experts, with prior studies reporting meta-cognitive pitfalls that affect novices' ability to verify outputs and prompt effectively. We focus on a task domain, machine learning (ML), which embodies both high complexity and low verifiability to understand the impact of LLM assistance on novices. Provided a buggy ML script and open access to ChatGPT, we conduct a formative study with eight novice ML engineers to understand their reliance on, interactions with, and perceptions of the LLM. We find that user actions can be roughly categorized into leading the LLM and led-by the LLM, and further investigate how they affect reliance outcomes like over- and under-reliance. These results have implications on novices' cognitive engagement in LLM-assisted tasks and potential negative effects on downstream learning. Lastly, we pose potential augmentations to the novice-LLM interaction paradigm to promote cognitive engagement.

Figures (1)

• Figure 1: Workflow analysis for each participant with reliance outcomes (top bar) and corresponding LLM actions (bottom bar). To disambiguate between the leading/led-by paradigm, the reliance outcome colours when the user lead the LLM are more saturated than when led-by the LLM.