Revisiting Method-Level Change Prediction: A Comparative Evaluation at Different Granularities

TL;DR

This study investigates whether method-level change prediction provides practical benefits over class-level prediction. By reproducing Farah et al.’s method-level approach and evaluating it at both granularity levels, plus an effort-aware perspective, the authors show that class-level predictions outperform method-level predictions when assessed at their native levels, but method-level predictions can surpass class-level performance when evaluated at method-level or under constrained maintenance effort. The work demonstrates a clear trade-off: finer granularity yields actionable change locations but can reduce overall predictive accuracy, while efficiency-focused evaluation reveals method-level predictions offer more efficient maintenance opportunities when resources are scarce. The findings inform guidelines on when to favor method-level versus class-level predictions in real-world maintenance planning and provide publicly available datasets for replication.

Abstract

To improve the efficiency of software maintenance, change prediction techniques have been proposed to predict frequently changing modules. Whereas existing techniques focus primarily on class-level prediction, method-level prediction allows for more direct identification of change locations. Method-level prediction can be useful, but it may also negatively affect prediction performance, leading to a trade-off. This makes it unclear which level of granularity users should select for their predictions. In this paper, we evaluated the performance of method-level change prediction compared with that of class-level prediction from three perspectives: direct comparison, method-level comparison, and maintenance effort-aware comparison. The results from 15 open source projects show that, although method-level prediction exhibited lower performance than class-level prediction in the direct comparison, method-level prediction outperformed class-level prediction when both were evaluated at method-level, leading to a median difference of 0.26 in accuracy. Furthermore, effort-aware comparison shows that method-level prediction performed significantly better when the acceptable maintenance effort is little.

Figures (8)

• Figure 1: Changes between releases.
• Figure 2: Overview of prediction technique.
• Figure 3: Difference in the distribution of change-prone methods between the datasets by Farah et al. and Catolino et al.
• Figure 4: Results of replication experiments.
• Figure 5: Results for RQ${}_{\mathrm{1}}$: direct comparison.