Ivie: Lightweight Anchored Explanations of Just-Generated Code

TL;DR

Ivie tackles the problem of understanding AI-generated code by introducing anchored, lightweight explanations that appear in-situ next to code in the editor. It is implemented as a VS Code extension and uses an LLM to segment code into expressions and blocks, producing brief explanations that are easy to access and dismiss. An in-lab study comparing Ivie to a chat-based AI baseline shows Ivie improves comprehension (accuracy and speed) while reducing perceived workload and distraction, with users valuing it as a complementary tool. The work highlights design implications for instructive copilots and suggests extensions to expandable, adaptable, and controllable explanations beyond the code editor.

Abstract

Programming assistants have reshaped the experience of programming into one where programmers spend less time writing and more time critically examining code. In this paper, we explore how programming assistants can be extended to accelerate the inspection of generated code. We introduce an extension to the programming assistant called Ivie, or instantly visible in-situ explanations. When using Ivie, a programmer's generated code is instantly accompanied by explanations positioned just adjacent to the code. Our design was optimized for extremely low-cost invocation and dismissal. Explanations are compact and informative. They describe meaningful expressions, from individual variables to entire blocks of code. We present an implementation of Ivie that forks VS Code, applying a modern LLM for timely segmentation and explanation of generated code. In a lab study, we compared Ivie to a contemporary baseline tool for code understanding. Ivie improved understanding of generated code, and was received by programmers as a highly useful, low distraction, desirable complement to the programming assistant.

Figures (9)

• Figure 1: The implementation of an instructive copilot for programming. Ivie creates interactive overlays that explain suggestions made by a programming assistant. When the programming assistant (e.g., Copilot) displays the suggestion, Ivie submits that suggestion in a prompt to an LLM, requesting that the code be segmented and explained. Ivie then integrates the explanations into the editor as overlays beneath the expressions they explain.
• Figure 2: A prompt for requesting expression-level explanations of generated code. This prompt requests explanations of suggested code. It provides a single example of how it would like code suggestions to be split into expressions with accompanying brief explanations of those expressions.
• Figure 3: Explaining expressions. After a programming assistant suggests code, Ivie fetches explanations of that code. If the suggestion consists of a single line, Ivie reveals explanations of meaningful expressions within that line, such as function calls, and parameters for those calls. The purpose of these explanations is to make explicit the intent of code that may not be self-evident (as might be the case for the programmer seeking to understand the precise behavior of the suggested table merge operation above).
• Figure 4: Explaining multi-line suggestions. When a programming assistant suggests multiple lines of code, Ivie splits up and explains that code. Its explanations appear in the right margin of the editor. The explanations are meant to help a programmer get a high-level understanding of the code. In the pictured scenario, these explanations might help the programmer understand that the two longest sections of the code suggestion configure each of two subplots, each with a different slice of the data.
• Figure 5: Comprehension results. Each data point for a box plot corresponds to a programmer's score on the assessment or their average question response time. Data is clustered by which tool (Ivie or baseline) was used for the programming task (Task A or B) associated with the assessment. For correctness, higher values are preferred. For time, lower values are preferred. The differences of Ivie vs. chat baseline on both correctness ($p<0.001$) and time ($p = 0.011$) are statistically significant.