SemOpt: LLM-Driven Code Optimization via Rule-Based Analysis

TL;DR

SemOpt addresses the challenge of reliably optimizing large software systems with LLMs by bypassing brittle retrieval of similar code and instead using static program analysis to locate optimizable regions, summarizing optimization strategies from real commits, and generating strategy-specific Semgrep rules to guide targeted prompts. The approach builds a clustered strategy library, uses Semgrep to detect eligible locations, and prompts LLMs to produce optimized code for each location, achieving better generalization across codebases and reduced optimization overhead. Empirical results on a 151-task benchmark and five real-world C/C++ projects show substantial improvements over baselines (up to 27x more successful optimizations on the benchmark and up to 218.07% speedups in the wild), with high semantic preservation and complementary gains when combined with RAG. SemOpt demonstrates practical scalability for large-scale software, offering a principled, analysis-driven path for LLM-assisted performance optimization.

Abstract

Automated code optimization aims to improve performance in programs by refactoring code, and recent studies focus on utilizing LLMs for the optimization. Typical existing approaches mine optimization commits from open-source codebases to construct a large-scale knowledge base, then employ information retrieval techniques such as BM25 to retrieve relevant optimization examples for hotspot code locations, thereby guiding LLMs to optimize these hotspots. However, since semantically equivalent optimizations can manifest in syntactically dissimilar code snippets, current retrieval methods often fail to identify pertinent examples, leading to suboptimal optimization performance. This limitation significantly reduces the effectiveness of existing optimization approaches. To address these limitations, we propose SemOpt, a novel framework that leverages static program analysis to precisely identify optimizable code segments, retrieve the corresponding optimization strategies, and generate the optimized results. SemOpt consists of three key components: (1) A strategy library builder that extracts and clusters optimization strategies from real-world code modifications. (2) A rule generator that generates Semgrep static analysis rules to capture the condition of applying the optimization strategy. (3) An optimizer that utilizes the strategy library to generate optimized code results. All the three components are powered by LLMs. On our benchmark containing 151 optimization tasks, SemOpt demonstrates its effectiveness under different LLMs by increasing the number of successful optimizations by 1.38 to 28 times compared to the baseline. Moreover, on popular large-scale C/C++ projects, it can improve individual performance metrics by 5.04% to 218.07%, demonstrating its practical utility.

Figures (5)

• Figure 1: The overview of SemOpt.
• Figure 2: The motivating example that leverages short-circuit evaluation to optimize the performance.
• Figure 3: The prompt for generating optimization results.
• Figure 4: Code optimization example for RQ1.
• Figure 5: Code optimization example for RQ3.