"Like a Nesting Doll": Analyzing Recursion Analogies Generated by CS Students using Large Language Models

TL;DR

This study investigates how first-year CS students use ChatGPT to generate recursion analogies, focusing on whether explicitly prompting for a personal topic increases analogy diversity and learning engagement. Analyzing 841 course responses, with 385 containing topic prompts, the authors code analogies into 29 themes and assess diversity with inter-rater reliability around 0.83–0.84. Key findings show that student-specified topics yield greater topic diversity (0.747) and longer explanations than GPT-selected prompts (0.284), and students report educational value and personal relevance with predominantly positive sentiment. The work highlights the potential of integrating LLMs for personalized, culturally relevant learning aids in computing education, while acknowledging limitations such as missing prompts, single-task design, and the need for further measurement of actual learning gains.

Abstract

Grasping complex computing concepts often poses a challenge for students who struggle to anchor these new ideas to familiar experiences and understandings. To help with this, a good analogy can bridge the gap between unfamiliar concepts and familiar ones, providing an engaging way to aid understanding. However, creating effective educational analogies is difficult even for experienced instructors. We investigate to what extent large language models (LLMs), specifically ChatGPT, can provide access to personally relevant analogies on demand. Focusing on recursion, a challenging threshold concept, we conducted an investigation analyzing the analogies generated by more than 350 first-year computing students. They were provided with a code snippet and tasked to generate their own recursion-based analogies using ChatGPT, optionally including personally relevant topics in their prompts. We observed a great deal of diversity in the analogies produced with student-prescribed topics, in contrast to the otherwise generic analogies, highlighting the value of student creativity when working with LLMs. Not only did students enjoy the activity and report an improved understanding of recursion, but they described more easily remembering analogies that were personally and culturally relevant.

Figures (3)

• Figure 1: Function to print a string in reverse, for which students were asked to generate an analogy.
• Figure 2: Instructions provided to students for Exercise 1.
• Figure 3: An Example Analogy Generated by P242