Protean Compiler: An Agile Framework to Drive Fine-grain Phase Ordering

TL;DR

The paper addresses the enduring phase-ordering problem in compilers by introducing Protean Compiler, an end-to-end, LLVM-integrated framework that enables fine-grained phase-ordering driven by an agile optimization loop. It combines an agile driver, a transformer-based IR2Score performance predictor, and a rich Protean Feature Set to iteratively search for optimal optimization recipes at module, function, or loop scopes, using simulated annealing to navigate the large optimization space. Key contributions include a fully integrated LLVM workflow, a 141-feature library, a clustering-based subsequence approach, and empirical results showing average runtime speedups up to 4.1% (and up to 15.7% on select cases), with modest build-time overhead; enhanced results are demonstrated when integrating with third-party ML frameworks and LLMs, achieving up to 10.1% and 8.5% improvements in targeted workloads. The work demonstrates practical impact by enabling faster-compiled binaries in real-world benchmarks, offering a path toward broader adoption via open-source release and extensible integration with ML and LLM ecosystems for automated, scalable phase ordering.

Abstract

The phase ordering problem has been a long-standing challenge since the late 1970s, yet it remains an open problem due to having a vast optimization space and an unbounded nature, making it an open-ended problem without a finite solution, one can limit the scope by reducing the number and the length of optimizations. Traditionally, such locally optimized decisions are made by hand-coded algorithms tuned for a small number of benchmarks, often requiring significant effort to be retuned when the benchmark suite changes. In the past 20 years, Machine Learning has been employed to construct performance models to improve the selection and ordering of compiler optimizations, however, the approaches are not baked into the compiler seamlessly and never materialized to be leveraged at a fine-grained scope of code segments. This paper presents Protean Compiler: An agile framework to enable LLVM with built-in phase-ordering capabilities at a fine-grained scope. The framework also comprises a complete library of more than 140 handcrafted static feature collection methods at varying scopes, and the experimental results showcase speedup gains of up to 4.1% on average and up to 15.7% on select Cbench applications wrt LLVM's O3 by just incurring a few extra seconds of build time on Cbench. Additionally, Protean compiler allows for an easy integration with third-party ML frameworks and other Large Language Models, and this two-step optimization shows a gain of 10.1% and 8.5% speedup wrt O3 on Cbench's Susan and Jpeg applications. Protean compiler is seamlessly integrated into LLVM and can be used as a new, enhanced, full-fledged compiler. We plan to release the project to the open-source community in the near future.

Figures (5)

• Figure 1: Protean Compiler System Architecture
• Figure 2: IR2Score Model Data Representation
• Figure 3: PFS and IR2VEC Feature Correlation with our Label (Module Speedup against O3)
• Figure 4: IR2Score Model Training and Validation
• Figure 5: Protean Compiler Integration Examples