AI-powered Code Review with LLMs: Early Results
Zeeshan Rasheed, Malik Abdul Sami, Muhammad Waseem, Kai-Kristian Kemell, Xiaofeng Wang, Anh Nguyen, Kari Systä, Pekka Abrahamsson
TL;DR
The paper tackles improving code review with large language models by proposing a four-agent LLM-based framework (Code Review, Bug Report, Code Smell, Code Optimization) coordinated via a central hub and powered by GPT-4. It demonstrates preliminary evidence that autonomous, multi-agent analysis can identify bugs, smells, and optimization opportunities beyond traditional static analysis, while also supporting developer education. The work lays out a clear path for evaluating documentation-generation efficacy, expanding to broader technical debt, and measuring educational impact, with the goal of streamlining the software development lifecycle. Overall, the approach represents a proactive, educative enhancement to code review that could improve software quality and developer proficiency at scale.
Abstract
In this paper, we present a novel approach to improving software quality and efficiency through a Large Language Model (LLM)-based model designed to review code and identify potential issues. Our proposed LLM-based AI agent model is trained on large code repositories. This training includes code reviews, bug reports, and documentation of best practices. It aims to detect code smells, identify potential bugs, provide suggestions for improvement, and optimize the code. Unlike traditional static code analysis tools, our LLM-based AI agent has the ability to predict future potential risks in the code. This supports a dual goal of improving code quality and enhancing developer education by encouraging a deeper understanding of best practices and efficient coding techniques. Furthermore, we explore the model's effectiveness in suggesting improvements that significantly reduce post-release bugs and enhance code review processes, as evidenced by an analysis of developer sentiment toward LLM feedback. For future work, we aim to assess the accuracy and efficiency of LLM-generated documentation updates in comparison to manual methods. This will involve an empirical study focusing on manually conducted code reviews to identify code smells and bugs, alongside an evaluation of best practice documentation, augmented by insights from developer discussions and code reviews. Our goal is to not only refine the accuracy of our LLM-based tool but also to underscore its potential in streamlining the software development lifecycle through proactive code improvement and education.