HistoryFinder: Advancing Method-Level Source Code History Generation with Accurate Oracles and Enhanced Algorithm

TL;DR

This work addresses the critical need for accurate method-level change histories by constructing two high-quality benchmarks (the corrected CodeShovel oracle and a HistoryFinder oracle) through automated synthesis from multiple tools plus expert validation. It then introduces HistoryFinder, a heuristic method-history generator that improves on prior approaches by handling merges and formatting changes while maintaining competitive runtimes. Comprehensive evaluation on 400 methods across 40 open-source projects shows HistoryFinder achieving the best precision, recall, and F1 across the new benchmarks, outperforming CodeShovel, CodeTracker, and Git-based baselines. The authors provide GUI, CLI, and Java library support to facilitate adoption and offer public access to the two oracles and the HistoryFinder tool, enabling researchers and practitioners to perform reliable method-level history analysis.

Abstract

Reconstructing a method's change history efficiently and accurately is critical for many software engineering tasks, including maintenance, refactoring, and comprehension. Despite the availability of method history generation tools such as CodeShovel and CodeTracker, existing evaluations of their effectiveness are limited by inaccuracies in the ground truth oracles used. In this study, we systematically construct two new oracles -- the corrected CodeShovel oracle and a newly developed HistoryFinder oracle -- by combining automated analysis with expert-guided manual validation. We also introduce HistoryFinder, a new method history generation tool designed to improve not only the accuracy and completeness of method change histories but also to offer competitive runtime performance. Through extensive evaluation across 400 methods from 40 open-source repositories, we show that HistoryFinder consistently outperforms CodeShovel, CodeTracker, IntelliJ, and Git-based baselines in terms of precision, recall, and F1 score. Moreover, HistoryFinder achieves competitive runtime performance, offering the lowest mean and median execution times among all the research-based tools. While Git-based tools exhibit the fastest runtimes, this efficiency comes at the cost of significantly lower precision and recall -- leaving HistoryFinder as the best overall choice when both accuracy and efficiency are important. To facilitate adoption, we provide a web interface, CLI, and Java library for flexible usage.

Figures (2)

• Figure 1: The process to generate accurate oracles. We first took the union of the change commits reported by all tools. The union of all commits was then manually verified by two authors to remove the false positives. The authors also added any missing commits they found that were not captured by any of the tools.
• Figure 2: Comparison of execution times across three oracles for five tools. To enhance readability, the y-axis is limited to 20 seconds for the first two subplots (CodeShovel oracles) and 50 seconds for the third subplot (HistoryFinder oracle).