Generation Probabilities Are Not Enough: Uncertainty Highlighting in AI Code Completions

TL;DR

This work tackles how to help programmers oversee AI-powered code completions by highlighting uncertainty. It compares two notions of uncertainty—generation probability and an edit-based likelihood—using a mixed-methods, within-subjects study with 30 programmers. The key finding is that highlighting tokens likely to be edited (edit-based uncertainty) speeds up task completion and yields more targeted edits, while highlighting low-probability tokens from the generator provides no notable benefit over no highlights. The study also maps a design space emphasizing granular, interpretable, and non-overwhelming cues, while acknowledging limitations around open-world applicability and potential automation bias. Overall, the results advocate prioritizing edit-driven uncertainty signaling over generation-probability cues in code-completion interfaces to support effective human-AI collaboration.

Abstract

Large-scale generative models enabled the development of AI-powered code completion tools to assist programmers in writing code. However, much like other AI-powered tools, AI-powered code completions are not always accurate, potentially introducing bugs or even security vulnerabilities into code if not properly detected and corrected by a human programmer. One technique that has been proposed and implemented to help programmers identify potential errors is to highlight uncertain tokens. However, there have been no empirical studies exploring the effectiveness of this technique -- nor investigating the different and not-yet-agreed-upon notions of uncertainty in the context of generative models. We explore the question of whether conveying information about uncertainty enables programmers to more quickly and accurately produce code when collaborating with an AI-powered code completion tool, and if so, what measure of uncertainty best fits programmers' needs. Through a mixed-methods study with 30 programmers, we compare three conditions: providing the AI system's code completion alone, highlighting tokens with the lowest likelihood of being generated by the underlying generative model, and highlighting tokens with the highest predicted likelihood of being edited by a programmer. We find that highlighting tokens with the highest predicted likelihood of being edited leads to faster task completion and more targeted edits, and is subjectively preferred by study participants. In contrast, highlighting tokens according to their probability of being generated does not provide any benefit over the baseline with no highlighting. We further explore the design space of how to convey uncertainty in AI-powered code completion tools, and find that programmers prefer highlights that are granular, informative, interpretable, and not overwhelming.

Figures (10)

• Figure 1: Overview of two approaches to uncertainty highlighting explored in our study: directly predicting the likelihood of an edit (top path), and using generation probabilities (bottom path). In this example, edit model-based highlighting points the programmer to the location of a necessary change (the list comprehension which erroneously has "1" in place of "-1"). The generation probability-based highlighting points the programmer to the function name, which, while being high variance, does not necessarily have any errors.
• Figure 2: Highlights in the Generation Probability condition (left) and the Edit Model condition (right) for the three coding tasks. The No Highlights condition displays the code completion alone.
• Figure 3: Screenshot of the interface used in our study. The "Task Overview" section provides general instructions for the task and interface. The "Coding Task" section provides a description of the current coding task. The "Expected Format" section describes the code format (e.g., function names and definitions) expected by unit tests. The "Suggestions" section provides some suggestions for debugging in our interface. Participants have the option to run their code as a file ("Run your code") or run our unit tests on their code ("Run unit tests"). When satisfied with their code, they can select "Submit."
• Figure 4: Average time taken per task in minutes (capped at 10) by highlight condition and coding task.
• Figure 5: Average accuracy on unit tests by highlight condition and coding task.