Hercules: A Compiler for Productive Programming of Heterogeneous Systems

TL;DR

This work tackles the productivity challenge of programming heterogeneous systems by decoupling application logic from hardware-specific optimizations through Juno, a hardware-agnostic imperative application language, and Hercules, a compiler with a fine-grained scheduling language and a Sea of Nodes IR. The Hercules IR uses explicit fork-join parallelism and value semantics to transform sequential code into device-targeted parallel code, enabling automatic device partitioning, code motion, and efficient generation of CPU and GPU kernels. Empirical evaluation across seven benchmarks shows Hercules delivering substantial speedups over both hand-written baselines ($1.26\times$ CPU, $1.47\times$ GPU) and the HPVM system ($9.31\times$ CPU, $16.18\times$ GPU), while the scheduling language provides clear, reusable patterns for reductions, tiling, and multi-core parallelism. Overall, the approach demonstrates productive separation of concerns between application development and performance optimization, with practical impact for portable, high-performance heterogeneous computing.

Abstract

Modern computing systems increasingly rely on composing heterogeneous devices to improve performance and efficiency. Programming these systems is often unproductive: algorithm implementations must be coupled to system-specific logic, including device-specific optimizations, partitioning, and inter-device communication and synchronization, which requires developing different programs for different system configurations. We propose the Juno language, which represents general purpose applications in an imperative form that can be transformed into parallel, optimized, system-specific code using an expressive and granular imperative scheduling language. We also introduce the Hercules compiler, which uses a novel intermediate representation to represent general and device-specific parallel code in a manner that is easy to analyze and manipulate using schedules. Our system achieves competitive performance with hand-optimized device-specific code (geomean speedups of $1.25\times$ and $1.48\times$ on the CPU and GPU) and significantly outperforms a prior general purpose heterogeneous programming system (geomean speedups of $9.31\times$ and $16.18\times$ on the CPU and GPU).

Figures (10)

• Figure 1: A Juno program to compute matrix multiplication.
• Figure 2: A schedule to tile and parallelize the matrix multiplication in Figure \ref{['fig:juno_matmul']}.
• Figure 3: The loop structure of matmul as it is scheduled. Blue blocks refer to the @inner loop, green blocks refer to the @middle loop, and red blocks refer to the @outer loop.
• Figure 4: A Hercules IR function that adds all the elements in an array. Control nodes are red and data nodes are blue. Data dependencies are solid, control flow edges are dashed, control-data interactions are dotted.
• Figure 5: A Hercules IR function performing matrix multiplication using fork-joins.