Using Ensemble Inference to Improve Recall of Clone Detection
Gul Aftab Ahmed, James Vincent Patten, Yuanhua Han, Guoxian Lu, David Gregg, Jim Buckley, Muslim Chochlov
TL;DR
This study addresses the recall limitations of large-scale code clone detection by evaluating four transformer-based neural networks (ADA, CodeBERT, GraphCodeBERT, CodeT5) and exploring ensemble inference within the SSCD framework. By embedding code fragments and performing approximate nearest-neighbor search, the approach scales to large codebases while focusing on maximizing recall with acceptable precision. Results show that ensembles consistently improve recall over individual models, with ADA delivering the best single-model performance but privacy constraints motivating high-recall non-cloud alternatives like CodeT5 and non-proprietary ensembles (e.g., CT5_CBF_GCB). The findings offer practical guidance for practitioners balancing recall, precision, and data confidentiality in industrial settings, and point to future work on broader benchmarks and additional ANN models.
Abstract
Large-scale source-code clone detection is a challenging task. In our previous work, we proposed an approach (SSCD) that leverages artificial neural networks and approximates nearest neighbour search to effectively and efficiently locate clones in large-scale bodies of code, in a time-efficient manner. However, our literature review suggests that the relative efficacy of differing neural network models has not been assessed in the context of large-scale clone detection approaches. In this work, we aim to assess several such models individually, in terms of their potential to maximize recall, while preserving a high level of precision during clone detection. We investigate if ensemble inference (in this case, using the results of more than one of these neural network models in combination) can further assist in this task. To assess this, we employed four state-of-the-art neural network models and evaluated them individually/in combination. The results, on an illustrative dataset of approximately 500K lines of C/C++ code, suggest that ensemble inference outperforms individual models in all trialled cases, when recall is concerned. Of individual models, the ADA model (belonging to the ChatGPT family of models) has the best performance. However commercial companies may not be prepared to hand their proprietary source code over to the cloud, as required by that approach. Consequently, they may be more interested in an ensemble-combination of CodeBERT-based and CodeT5 models, resulting in similar (if slightly lesser) recall and precision results.