iCodeReviewer: Improving Secure Code Review with Mixture of Prompts

TL;DR

This paper defines secure code review as a multi-goal task and introduces iCodeReviewer, a large-language-model (LLM) based approach that uses a novel mixture-of-prompts architecture to improve coverage and precision in security issue identification and localization. A four-phase pipeline combines AST-based feature extraction, a routing algorithm that activates only relevant prompt experts, parallel issue identification, and structured review generation, with a multi-proposition determination strategy to mitigate LLM hallucinations. Evaluations on an internal dataset show significant gains in F1 for issue identification ($F1=63.98\%$) and localization accuracy ($ACC=47.58\%$) over baselines, along with high review helpfulness and strong production acceptance rates (up to $84\%$). The work demonstrates practical impact through deployment in production lines and provides a scalable path for adding new security issues via additional prompt experts. Overall, iCodeReviewer advances secure code review by combining targeted expert prompts with dynamic routing to balance precision and coverage in real-world software development.

Abstract

Code review is an essential process to ensure the quality of software that identifies potential software issues at an early stage of software development. Among all software issues, security issues are the most important to identify, as they can easily lead to severe software crashes and service disruptions. Recent research efforts have been devoted to automated approaches to reduce the manual efforts required in the secure code review process. Despite the progress, current automated approaches on secure code review, including static analysis, deep learning models, and prompting approaches, still face the challenges of limited precision and coverage, and a lack of comprehensive evaluation. To mitigate these challenges, we propose iCodeReviewer, which is an automated secure code review approach based on large language models (LLMs). iCodeReviewer leverages a novel mixture-of-prompts architecture that incorporates many prompt experts to improve the coverage of security issues. Each prompt expert is a dynamic prompt pipeline to check the existence of a specific security issue. iCodeReviewer also implements an effective routing algorithm to activate only necessary prompt experts based on the code features in the input program, reducing the false positives induced by LLM hallucination. Experiment results in our internal dataset demonstrate the effectiveness of iCodeReviewer in security issue identification and localization with an F1 of 63.98%. The review comments generated by iCodeReviewer also achieve a high acceptance rate up to 84% when it is deployed in production environments.

Figures (2)

• Figure 1: The overview of iCodeReviewer.
• Figure 2: The acceptance rate before and after deploying iCodeReviewer in two production lines.