SysLLMatic: Large Language Models are Software System Optimizers

TL;DR

SysLLMatic presents a principled, pattern-guided pipeline that leverages large language capabilities to automatically optimize real-world software. By combining a curated catalog of 43 performance patterns, principled hotspot identification, and a Diagnosis Cycle feedback loop, the system achieves substantial improvements in latency, throughput, energy, and memory across micro-benchmarks and large-scale DaCapo applications, often outperforming state-of-the-art baselines and surpassing compiler heuristics in real-world settings. The work also introduces a rigorous catalog construction process, a concrete prototype, and multi-objective evaluations that consider maintainability and resource costs, providing practical insight into when and how LLM-guided optimization can be deployed. While promising, the study also highlights limitations around correctness validation, context handling, and the computational and energy costs of optimization, underscoring the need for robust validation and scalable deployment strategies as push-button optimization advances. Overall, SysLLMatic lays a foundation for scalable, cross-language, real-world code optimization informed by system performance principles, with clear directions for reducing inference costs and improving correctness guarantees.

Abstract

Automatic software system optimization can improve software speed and save energy. Traditional approaches to optimization rely on manual tuning and compiler heuristics, limiting their ability to generalize across diverse codebases. Recent methods using LLMs introduce automation, but they do not scale effectively to the complexity and size of real-world software systems, leaving a gap in practical applicability. We present SysLLMatic, a system that integrates LLMs with performance diagnostics feedback and a curated catalog of 43 optimization patterns to automatically optimize software code. Our approach builds on recent advances in LLM-based code optimization and specifically targets the limitations of existing systems in handling real-world software applications. We evaluate it on three benchmark suites: HumanEval_CPP (competitive programming in C++), SciMark2 (scientific kernels in Java), and DaCapoBench (large-scale software systems in Java). Results show that SysLLMatic can improve software system performance, including latency, throughput, energy efficiency, memory usage, and CPU utilization. It consistently outperforms state-of-the-art LLM baselines on microbenchmarks. On large-scale application codes, to which prior LLM approaches have not scaled, it surpasses compiler optimizations, achieving average relative improvements of 1.5x in latency (vs. 1.01x for the compiler) and 1.76x in throughput (vs. 1.02x for the compiler). Our findings demonstrate that LLMs, guided by principled system thinking through the optimization pattern catalog and appropriate performance diagnostics, can serve as viable software system optimizers. We further identify limitations of our approach and the challenges involved in handling complex applications. This work provides a foundation for generating optimized code across various languages, benchmarks, and program sizes in a principled manner.

Figures (16)

• Figure 1: The Diagnosis Cycle Gregg_2021 for performance optimization, consisting of forming a hypothesis, instrumenting the program, and collecting data. The process is iterative: data informs whether the hypothesis holds, and new hypotheses are refined and tested in subsequent cycles.
• Figure 2: Construction of the Performance Optimization Pattern Catalog: We annotate, merge, and filter patterns from literature to derive 43 unique optimization patterns across 7 themes.
• Figure 3: Optimization patterns by 7 themes. For readability, pattern names are shortened and in some cases merged to conserve space. Symbols indicate four change types: Filter/Remove code, Rearrange code, Energy-focused changes, and Rewrite/Modify code.
• Figure 4: Overview of SysLLMatic, our automated LLM-based optimization framework. SysLLMatic integrates domain-specific performance knowledge and follows an iterative diagnosis cycle including generation, evaluation, and feedback to optimize performance. Numbered circles denote major components of SysLLMatic. Component ① encompasses three distinct LLM roles. Component ② is Code Analysis, and Component ③ is Correctness Check.
• Figure 5: Hotspot identification pipeline and example profiling output. Dynamic analysis collects stack traces with Flame Graphs, which are aggregated to rank functions by execution frequency (left shows an excerpt from BioJava with counts). The Top-$K$ hotspots are then extracted through static analysis to form a candidate set of performance bottlenecks. This process ensures that optimization focuses on code where performance cost is most concentrated.