Evaluating the Performance of Clone Detection Tools in Detecting Cloned Co-change Candidates

TL;DR

This study addresses the problem of identifying cloned co-change candidates during software evolution to support change impact analysis. It compares 12 clone-detection configurations across eight open-source C/Java projects, using ground-truth CCC derived from Unix diffs and cross-tool unions to compute Recall, Precision, and F1, culminating in a final detector ranking. The results show that pattern-based Type-3 configurations in CloneWorks, along with Deckard and CCFinder, reliably identify cloned co-change candidates, while text-based detectors underperform; performance correlates with clone fragment count and line coverage. The findings provide actionable guidelines for selecting and configuring clone detectors in maintenance contexts and introduce a new maintenance-focused dimension to clone research.

Abstract

Co-change candidates are the group of code fragments that require a change if any of these fragments experience a modification in a commit operation during software evolution. The cloned co-change candidates are a subset of the co-change candidates, and the members in this subset are clones of one another. The cloned co-change candidates are usually created by reusing existing code fragments in a software system. Detecting cloned co-change candidates is essential for clone-tracking, and studies have shown that we can use clone detection tools to find cloned co-change candidates. However, although several studies evaluate clone detection tools for their accuracy in detecting cloned fragments, we found no study that evaluates clone detection tools for detecting cloned co-change candidates. In this study, we explore the dimension of code clone research for detecting cloned co-change candidates. We compare the performance of 12 different configurations of nine promising clone detection tools in identifying cloned co-change candidates from eight open-source C and Java-based subject systems of various sizes and application domains. A ranked list and analysis of the results provides valuable insights and guidelines into selecting and configuring a clone detection tool for identifying co-change candidates and leads to a new dimension of code clone research into change impact analysis.

Figures (5)

• Figure 1: This diagram explains how we calculated the Recall and Precision in one revision. We first identified all the cloned and dissimilar co-change candidates as ground truth using the Unix diff operation, then we have used the results of each of the clone detection tools to identify those cloned co-change candidates. True Positives (TP) are the co-changes that we successfully identified using the clone detectors. Our process also provided some False Positives (FP) in each revision. To separate the cloned co-change candidates from all the previously identified co-change candidates, we took the union of all the TPs from all the clone detection tools. The Recall is the ratio of TPs and the number of cloned co-change candidates (TP+FN), and the Precision is the ratio of TPs and the number of total co-change suggestions (TP+FP). We repeated the process for all the revisions of all the subject systems used in our investigation.
• Figure 2: Average recall of different tools
• Figure 3: Average precision of different tools
• Figure 4: Comparing unique line coverage by clone fragments and number of clone fragments from different clone detectors.
• Figure 5: Comparing Distribution of F1 Scores in Different Clone Detectors