Refactoring for Novices in Java: An Eye Tracking Study on the Extract vs. Inline Methods

TL;DR

Teachers should be cautious about premature modularization for novices and highlight eye tracking as a useful complement to static metrics, according to results of a controlled experiment with Java novices.

Abstract

Developers often extract methods to improve readability, understanding, and reuse, while inlining keeps logic in one block. Prior work based on static metrics has not shown clear differences between these practices, and the human side of comprehension and navigation remains underexplored. We investigate Inline Method vs. Extract Method refactorings using a dynamic approach: eye tracking while participants read and solve tasks. We analyze key code areas and compare visual effort and reading behavior (fixation duration and count, regressions, revisits), alongside time and attempts. We ran a controlled experiment with 32 Java novices, followed by short interviews. Each participant solved eight simple tasks across four programs presented in an inlined version and four in an extracted version. We also surveyed 58 additional novices for complementary quantitative and qualitative data. Results show that effects depend on task difficulty. In two tasks, method extraction improved performance and reduced visual effort, with time decreasing by up to 78.8% and regressions by 84.6%. For simpler tasks (e.g., square area), extraction hurt performance: time increased by up to 166.9% and regressions by 200%. Even with meaningful method names, novices often switched back and forth between call sites and extracted methods, increasing navigation and cognitive load. Preferences frequently favored extraction for readability and reuse, but did not always match measured performance. These findings suggest educators should be cautious about premature modularization for novices and highlight eye tracking as a useful complement to static metrics.

Figures (31)

• Figure 1: Gaze patterns for (a) Inline Method version of the code that calculates the factorial of a number, and (b) Extract Method version.
• Figure 2: Gaze patterns for (a) Inline Method version of the code that calculates the area of a square, and (b) Extract Method version of the code.
• Figure 3: Experiment sections divided into five sequential steps.
• Figure 4: Programs evaluated: Sum Numbers, Calculate Next Prime, Return Highest Grade, Calculate Factorial, Count Multiples of Three, Calculate Area of a Square, Check If Even, and Count Number of Digits. Shaded areas represent our AOIs.
• Figure 5: Structure of the experiment in terms of experimental units divided into 16 squares. Subject$_1$ takes four programs (P$_1$--P$_4$) with the Extract Method (E) of Sum Numbers (P$_1$), Calculate Next Prime (P$_2$), Return Highest Grade (P$_3$), Calculate Factorial (P$_4$). These programs are from set of programs 1 (SP$_1$). Subject$_1$ also takes four programs (P$_5$--P$_{8}$) from the set of programs 2 (SP$_2$) comprising the Inline Method (I) of Count Multiples of Three (P$_5$), and Calculate Area of a Square (P$_6$), Check If Even (P$_7$), and Count Number of Digits (P$_8$). Subject$_2$ takes the complement to that. "Output" describes a task in which the subject has to specify the correct output.