Let's Ask AI About Their Programs: Exploring ChatGPT's Answers To Program Comprehension Questions

TL;DR

The paper investigates how state-of-the-art LLMs (GPT-3.5 and GPT-4) answer questions about code they generate, focusing on program comprehension QLCs. It uses six CodeCheck Python exercises, generates multiple LLM-produced solutions, constructs QLCs from those solutions with QLCpy, and has both models answer the QLCs before detailed manual analysis of correctness and error types. Results show GPT-4 outperforms GPT-3.5 across most QLC types, but both models exhibit non-trivial mistakes (e.g., illogical execution, line-number errors, and occasional hallucinations), indicating limits to AI-assisted program understanding. The work discusses implications for education, such as using LLMs to model student behavior and to inform prompting strategies, while acknowledging limitations and proposing directions for future research on learning outcomes and prompt design.

Abstract

Recent research has explored the creation of questions from code submitted by students. These Questions about Learners' Code (QLCs) are created through program analysis, exploring execution paths, and then creating code comprehension questions from these paths and the broader code structure. Responding to the questions requires reading and tracing the code, which is known to support students' learning. At the same time, computing education researchers have witnessed the emergence of Large Language Models (LLMs) that have taken the community by storm. Researchers have demonstrated the applicability of these models especially in the introductory programming context, outlining their performance in solving introductory programming problems and their utility in creating new learning resources. In this work, we explore the capability of the state-of-the-art LLMs (GPT-3.5 and GPT-4) in answering QLCs that are generated from code that the LLMs have created. Our results show that although the state-of-the-art LLMs can create programs and trace program execution when prompted, they easily succumb to similar errors that have previously been recorded for novice programmers. These results demonstrate the fallibility of these models and perhaps dampen the expectations fueled by the recent LLM hype. At the same time, we also highlight future research possibilities such as using LLMs to mimic students as their behavior can indeed be similar for some specific tasks.

Figures (6)

• Figure 1: Outline of our methodology: (1) We use LLM to generate different programs solving the same task for 6 exercises, (2) We generate QLCs for each program code, (3) We prompt LLM to answer the QLCs for each program generated earlier. This is followed by a manual analysis, discussed in Section \ref{['subsec:answer-analysis']}.
• Figure 2: The employed QLC types are marked with ordinal numbers and situated by the area of program comprehension they target in Schulte's Block Model schulte2008blockmodel
• Figure 3: Success rates by QLC type and programming task for the two LLMs --- Darker red marks lower success.
• Figure 4: Number of incorrect answers by QLC type and error code for the two LLMs --- Darker red marks higher frequency of the error code.
• Figure 5: An incorrect answer coded as h. misconception about code element: In the program, sum is a built-in function, not a variable.