RESTORE: Retrospective Fault Localization Enhancing Automated Program Repair

TL;DR

Experiments involving faults from the Defects4J standard benchmark indicate that retrospective fault localization can boost automated program repair: Restore efficiently explores a large fix space, delivering state-of-the-art effectiveness while simultaneously boosting performance.

Abstract

Fault localization is a crucial step of automated program repair, because accurately identifying program locations that are most closely implicated with a fault greatly affects the effectiveness of the patching process. An ideal fault localization technique would provide precise information while requiring moderate computational resources---to best support an efficient search for correct fixes. In contrast, most automated program repair tools use standard fault localization techniques---which are not tightly integrated with the overall program repair process, and hence deliver only subpar efficiency. In this paper, we present retrospective fault localization: a novel fault localization technique geared to the requirements of automated program repair. A key idea of retrospective fault localization is to reuse the outcome of failed patch validation to support mutation-based dynamic analysis---providing accurate fault localization information without incurring onerous computational costs. We implemented retrospective fault localization in a tool called RESTORE---based on the JAID Java program repair system. Experiments involving faults from the Defects4J standard benchmark indicate that retrospective fault localization can boost automated program repair: RESTORE efficiently explores a large fix space, delivering state-of-the-art effectiveness (41 Defects4J bugs correctly fixed, 8 more than any other automated repair tools for Java) while simultaneously boosting performance (speedup over 3 compared to JAID). Retrospective fault localization is applicable to any automated program repair techniques that rely on fault localization and dynamic validation of patches.

Figures (6)

• Figure 1: An overview of how [1]Restore works. [1]Restore can improve the performance of any generate-and-validate automated program repair tool. Such a tool inputs a faulty program and some test cases exercising the program. The first, crucial, step of fixing is fault localization, which determines a list of snapshots: program states that are indicative of error; for each suspicious snapshot, fix generation builds a number of candidate fixes of the input program by exploring a limited number of program mutations that may avoid the suspicious states; fix validation reruns the available tests on each candidate built by fix generation; only candidates that pass all tests are valid fixes, which are the tool's output to the user. [1]Restore kicks in during the first run of such a program repair tool, by introducing a feedback loop (in grey) that improves the effectiveness of fault localization. [1]Restore performs a partial fix validation, whose goal is quickly identifying candidate fixes that fail validation---which are treated as mutants of the input program; information about how mutants' behavior differ from the input program supports a mutation-based fault localization step that sharpens the identification of suspicious snapshots. As we demonstrate in \ref{['sec:evaluation']}, [1]Restore's feedback loop significantly improves effectiveness and efficiency of automated program repair.
• Figure 2: Schemas to build candidate fixes from a code snippet built from snapshot $\langle \ell, e, v \rangle$, where is the statement at $\ell$ in method under fixing.
• Figure 3: Visual explanation of linear regression lines. The two regression lines ${\color{kindA} y_1} = 130 + 0.5\,x$ and ${\color{kindB} y_2} = 30 + 0.5\,y$ have the same slope but different intercepts. Therefore, $y_2$ crosses the "no effect" line ${\color{cred} y_0} = x$ at $\bar{x}_2 = 60$, much earlier than $y_1$ that crosses it at $\bar{x}_1 = 260$. The crossing ratio scales the crossing coordinates $\bar{x}_1$ and $\bar{x}_2$ over the range of values on the $x$ axis. If the range is the whole $x$ axis from $0$ to $400$, the crossing ratios are simply $\chi_1 = \bar{x}_1/400 = 0.15$ and $\chi_2 = \bar{x}_2/400 = 0.65$, which indicate that $y_1$ is above $y_0$ for only 15% of the data, and $y_2$ for 65% of the data.
• Figure 4: Comparison of [1]Jaid and [1]Restore on various measures. For each measure $m$, a point with coordinates $x = J_{m, k}, y = R_{m, k}$ indicates that [1]Jaid costed $J_{m, k}$ of $m$ on fault $k$ while [1]Restore costed $R_{m, k}$ of $m$ on fault $k$. The dashed line is $y = x$; the solid line is the linear regression with $y$ dependent on $x$.
• Figure 5: Faults in [1]De-fects-4J bugs for which SimFix and SimFix+ can build correct fixes.