GRACE: Globally-Seeded Representation-Aware Cluster-Specific Evolution for Compiler Auto-Tuning

TL;DR

This work tackles the NP-hard problem of compiler phase-ordering and pass selection by introducing GRACE, a multi-stage framework that first seeds a reduced, high-quality search space using pass synergy, then learns program embeddings via contrastive learning to enable similarity-aware clustering. By performing cluster-specific coreset evolution with a guided genetic algorithm, GRACE produces a compact set of elite pass sequences tailored to program groups, which are then applied at test time with optional refinement steps to ensure robustness. Empirically, GRACE achieves state-of-the-art LLVM IR instruction-count reductions (approximately $10.1\%$ average over seven datasets) while keeping tuning times under $1$ second per program and demonstrating strong generalization to LLVM $18.1.6$. The approach offers practical deployment advantages, including interpretability of the coreset and robustness via an Oz baseline fallback, and it opens avenues for extension to multi-objective optimization and broader compiler ecosystems.

Abstract

Compiler pass selection and phase ordering present a significant challenge in achieving optimal program performance, particularly for objectives like code size reduction. Standard compiler heuristics offer general applicability but often yield suboptimal, program-specific results due to their one-size-fits-all nature. While iterative compilation can find tailored solutions, its prohibitive search cost limits practical use. Machine learning approaches promise faster inference but frequently struggle with generalization to unseen programs. This paper introduces GRACE, a novel framework for compiler auto-tuning, demonstrated for LLVM IR instruction count optimization. GRACE effectively curtails the search space by leveraging pass synergies and a weighted scoring method to generate initial high-quality candidate sequences and a pass pool. It then employs contrastive learning, using pass sequence-based data augmentation, to create program embeddings that facilitate similarity-aware clustering. Evolutionary search within these clusters yields a coreset of $k$ specialized pass sequences designed for robust generalization to unseen programs. At test time, GRACE efficiently selects the best coreset sequence and refines it using lightweight techniques. Experimental results on seven diverse datasets show that GRACE reduces LLVM IR instruction count by an average of 10.09% on LLVM 10.0.0 and 10.19% on LLVM 18.1.6 compared to opt -Oz, while incurring an average tuning time of less than 1s per program, demonstrating its state-of-the-art performance and practical effectiveness.

Figures (6)

• Figure 1: The GRACE framework.
• Figure 2: Distribution of weighted scores for all unique high-performing sequences. The spread indicates varying levels of optimization quality and reliability across candidate sequences, highlighting the diversity of promising optimization strategies uncovered by GRACE.
• Figure 3: t-SNE visualization of the learned program embeddings $e_k$.
• Figure 4: Distributions of Performance Characteristics for the $k$ Coreset Sequences. For each of the $k$ coreset sequences (one per cluster), we evaluate its performance on all programs within its respective cluster to derive three metrics: average LLVM IR instruction count reduction (Avg), standard deviation of these reductions (Std), and negative impact rate (NegRate) compared to opt -Oz. The histograms and smoothed curves presented here show the distributions of these $k$ individual Avg, Std, and NegRate values, offering insights into the overall performance profile of the generated coreset.
• Figure 5: GRACE’s performance and efficiency on LLVM 18.1.6 across seven benchmark suites. (a) Instruction count reduction relative to opt -Oz. (b) Average per-program tuning time, demonstrating low overhead.