Human Attention During Localization of Memory Bugs in C Programs

TL;DR

The paper investigates how programmers allocate visual attention when statically localizing memory bugs in C programs using eye-tracking, code exploration, and self-reports in a controlled lab with 21 participants across three programs. It reveals that most gaze turns are concentrated on a small subset of code (roughly 75% of fixations on 25% of functions) and that fixating on the correct line does not always yield correct localization, while successful bug localization shows higher regression and closer fixation patterns. The authors discuss practical implications for AI-assisted debugging and education, and provide data and code to support reproducibility. This work advances understanding of human factors in low-level bug localization and sets the stage for targeted tooling to assist developers.

Abstract

This paper presents a study of human visual attention during localization of memory bugs in C. Human visual attention refers to the mechanical processes by which we selectively process and prioritize information. Visual attention is important to study because it is central to what information people (who are sighted) use to solve a particular problem. Meanwhile, memory bugs are among the most common types of bugs in C programs that manifest as a variety of program faults. In this paper, we study human visual attention while people attempt to locate memory bugs in code. We recruit 21 programmers to locate between one and eight memory bugs in three C programs for 1.5-2 hours each. In total we collected observations of 31 hours of programmer effort. The bugs in our study cover memory leaks, overflows, and double frees, which are among the most common memory bugs. We analyze the task outcomes in terms of success rate and related factors, patterns of visual attention overall such as what lines and functions are read, and finally we explore differences of visual attention patterns during success versus failure cases.

Figures (13)

• Figure 1: This is the bug report for ladybug. This report is adapted from issue #6861 in the sway repository.
• Figure 2: The interface presented to participants. Area A is the bug report, area B is the code, area C is the file explorer, and area D is a text chat window with GPT-4o for AI help.
• Figure 3: Histograms of confidence, difficulty, and accuracy scores from all participants and tasks. Confidence and difficulty are self-reported by participants. Accuracy is graded on a rubric by the authors of this paper. "What" refers to the bug cause. "Where" refers to the bug location in code.
• Figure 4: Official fix for ladybug. See Figure \ref{['fig:ladybug_report']} for the ladybug bug report. Expected bug cause location is line 48 (before fix): free(seat_device);.
• Figure 5: Mean time required (in seconds) and mean scores per bug. Column "Issue Num" is the issue number in the project's bug tracking system, included here for reproducibility. Column "Codename" is a name we used in our experiments and in this paper's text for more memorable record-keeping.