Explaining Code with a Purpose: An Integrated Approach for Developing Code Comprehension and Prompting Skills

TL;DR

This paper tackles how to simultaneously develop code comprehension and prompting skills in the era of powerful language models. It introduces an EiPE-based assessment where students describe code in plain language to prompt an LLM to generate equivalent code, which is automatically graded via a test suite. In a CS1 course with approximately 900 students, the authors examine task success, prompt quality via the SOLO taxonomy, and student perceptions, finding high completion rates and a strong link between relational prompts and correctness. The results suggest a viable path to integrate code understanding and prompt design in programming education, while noting scalability and assessment reliability considerations.

Abstract

Reading, understanding and explaining code have traditionally been important skills for novices learning programming. As large language models (LLMs) become prevalent, these foundational skills are more important than ever given the increasing need to understand and evaluate model-generated code. Brand new skills are also needed, such as the ability to formulate clear prompts that can elicit intended code from an LLM. Thus, there is great interest in integrating pedagogical approaches for the development of both traditional coding competencies and the novel skills required to interact with LLMs. One effective way to develop and assess code comprehension ability is with ``Explain in plain English'' (EiPE) questions, where students succinctly explain the purpose of a fragment of code. However, grading EiPE questions has always been difficult given the subjective nature of evaluating written explanations and this has stifled their uptake. In this paper, we explore a natural synergy between EiPE questions and code-generating LLMs to overcome this limitation. We propose using an LLM to generate code based on students' responses to EiPE questions -- not only enabling EiPE responses to be assessed automatically, but helping students develop essential code comprehension and prompt crafting skills in parallel. We investigate this idea in an introductory programming course and report student success in creating effective prompts for solving EiPE questions. We also examine student perceptions of this activity and how it influences their views on the use of LLMs for aiding and assessing learning.

Figures (6)

• Figure 1: Explaining the purpose of a code fragment to an LLM until it generates functionally equivalent code, targeting code comprehension and prompt crafting skills in parallel.
• Figure 2: A problem presented in the form of a function implementation with obfuscated identifier names; students enter a natural language description that will be used to prompt an LLM to generate equivalent code to that shown.
• Figure 3: Distribution of prompt lengths for each of the two lab sessions (Lab B enforced a 255 character limit).
• Figure 4: Proportion of prompts generating correct and incorrect code at each SOLO level.
• Figure 5: Proportion of prompts classified at each SOLO-level for each lab.