LPR: Large Language Models-Aided Program Reduction

TL;DR

Program reduction must balance cross-language generality with language-specific semantics. LPR addresses this by alternately applying a language-generic reducer with LLM-driven semantic transformations and guiding prompts, enabling language-aware reductions across C, Rust, and JavaScript. Empirical results show LPR yields substantial program-size reductions and competitive efficiency, outperforming Vulcan and complementing C-Reduce when paired with Vulcan. The approach is extensible via new transformations and multi-level prompts, offering practical impact for compiler debugging and broader software-engineering tasks.

Abstract

Program reduction is a prevalent technique to facilitate compilers' debugging by automatically minimizing bug-triggering programs. Existing program reduction techniques are either generic across languages (e.g., Perses and Vulcan) or specifically customized for one certain language by employing language-specific features, like C-Reduce. However, striking the balance between generality across multiple programming languages and specificity to individual languages in program reduction is yet to be explored. This paper proposes LPR, the first technique utilizing LLMs to perform language-specific program reduction for multiple languages. The core insight is to utilize both the language-generic syntax level program reduction (e.g., Perses) and the language-specific semantic level program transformations learned by LLMs. Alternately, language-generic program reducers efficiently reduce programs into 1-tree-minimality, which is small enough to be manageable for LLMs; LLMs effectively transform programs via the learned semantics to expose new reduction opportunities for the language-generic program reducers to further reduce the programs. Our extensive evaluation on 50 benchmarks across three languages (C, Rust, and JavaScript) has highlighted LPR's practicality and superiority over Vulcan, the state-of-the-art language-generic program reducer. For effectiveness, LPR surpasses Vulcan by producing 24.93%, 4.47%, and 11.71% smaller programs on benchmarks in C, Rust and JavaScript. Moreover, LPR and Vulcan have demonstrated their potential to complement each other. By using Vulcan on LPR's output for C programs, we achieve program sizes comparable to those reduced by C-Reduce. For efficiency, LPR takes 10.77%, 34.88%, 36.96% less time than Vulcan to finish all benchmarks in C, Rust and JavaScript, separately.

Figures (6)

• Figure 1: Code snippet from LLVM-31259, showcasing the original code, the effectiveness of Loop Unrolling, and the final results by LPR, Vulcan and C-Reduce.
• Figure 2: The workflow of LPR.
• Figure 3: An example of prompt design. and denote system prompt and user prompt provided by the users. denotes the responses from the LLM.
• Figure 4: Program size changes induced by each transformation on benchmarks of Benchmark-C. In each subplot, the left box-plot and red dots represent how the size of each program changes before and after executing the transformation. The right box-plot and blue dots represent the size change of each benchmark after executing the transformation and the follow-up Perses reduction. There are a total of 20 benchmarks in Benchmark-C, and each experiment is repeated 5 times. Therefore, we draw 100 data points on each boxplot.
• Figure 5: Average size decrease and its percentage induced by each transformation within Benchmark-C.