Issue-Oriented Agent-Based Framework for Automated Review Comment Generation

TL;DR

RevAgent tackles the problem of generic, issue-agnostic code-review comments by introducing a three-stage, multi-agent framework where category-specific commentators generate candidate feedback, a critic selects the best aligned comment, and targeted fine-tuning with retrieval-based data enhances specialization. On Curev, RevAgent outperforms state-of-the-art PLM- and LLM-based baselines across BLEU, ROUGE-L, METEOR, and SBERT, while achieving strong issue-category prediction accuracy and favorable human judgments. The approach balances performance and efficiency, with detailed ablations confirming the importance of fine-tuning, specialized agents, and candidate retrieval. These results suggest that structured, issue-oriented collaboration among specialized agents can significantly improve both the quality and relevance of automated code review comments, offering practical benefits for large-scale software development. The work also identifies practical challenges, such as domain knowledge gaps and project-specific standards, and outlines directions for future improvements at the repository level and with more granular issue taxonomies.

Abstract

Code review (CR) is a crucial practice for ensuring software quality. Various automated review comment generation techniques have been proposed to streamline the labor-intensive process. However, existing approaches heavily rely on a single model to identify various issues within the code, limiting the model's ability to handle the diverse, issue-specific nature of code changes and leading to non-informative comments, especially in complex scenarios such as bug fixes. To address these limitations, we propose RevAgent, a novel agent-based issue-oriented framework, decomposes the task into three stages: (1) Generation Stage, where five category-specific commentator agents analyze code changes from distinct issue perspectives and generate candidate comments; (2) Discrimination Stage, where a critic agent selects the most appropriate issue-comment pair; and (3) Training Stage, where all agents are fine-tuned on curated, category-specific data to enhance task specialization. Evaluation results show that RevAgent significantly outperforms state-of-the-art PLM- and LLM-based baselines, with improvements of 12.90\%, 10.87\%, 6.32\%, and 8.57\% on BLEU, ROUGE-L, METEOR, and SBERT, respectively. It also achieves relatively higher accuracy in issue-category identification, particularly for challenging scenarios. Human evaluations further validate the practicality of RevAgent in generating accurate, readable, and context-aware review comments. Moreover, RevAgent delivers a favorable trade-off between performance and efficiency.

Figures (2)

• Figure 1: The overview of RevAgent
• Figure 2: Generated comments by RevAgent and baselines