Predicting Expert Evaluations in Software Code Reviews

TL;DR

The paper tackles the bottleneck of manual code reviews by introducing an algorithmic model calibrated to expert judgments that can predict coding time and implementation time from per-commit data. It employs static code analysis combined with a Git-based workflow and a random forest to weight features across Code Structure, Code Quality Metrics, Implementation Details, and Architectural Elements, emphasizing OO languages. The results show strong correlations with human judgments for time and some complexity measures ($r=0.82$ and $r=0.86$ for time; ICCs indicate solid agreement for key metrics), along with substantial speedups (under 1 second per commit). This approach offers a scalable, data-driven supplement to manual reviews, while recognizing limitations in subjective dimensions and the need for broader language generalization and validation across diverse development contexts.

Abstract

Manual code reviews are an essential but time-consuming part of software development, often leading reviewers to prioritize technical issues while skipping valuable assessments. This paper presents an algorithmic model that automates aspects of code review typically avoided due to their complexity or subjectivity, such as assessing coding time, implementation time, and code complexity. Instead of replacing manual reviews, our model adds insights that help reviewers focus on more impactful tasks. Calibrated using expert evaluations, the model predicts key metrics from code commits with strong correlations to human judgments (r = 0.82 for coding time, r = 0.86 for implementation time). By automating these assessments, we reduce the burden on human reviewers and ensure consistent analysis of time-consuming areas, offering a scalable solution alongside manual reviews. This research shows how automated tools can enhance code reviews by addressing overlooked tasks, supporting data-driven decisions and improving the review process.