Allo: A Programming Model for Composable Accelerator Design

TL;DR

Allo addresses the bottleneck of accelerator design by separating algorithm specification from hardware optimizations and enabling progressive, verifiable customization. It introduces decoupled hardware customization primitives, reusable parameterized kernels, and composable schedules that preserve hierarchical structure while enabling cross-kernel optimizations. A Python frontend with an MLIR dialect provides a practical, type-safe path from high-level code to hardware bitstreams, supported by a memory-layout type system and holistic dataflow optimizations. Empirical results on PolyBench demonstrate improvements over state-of-the-art HLS tools and ADLs, and experiments on GPT-2 show substantial performance and energy-efficiency gains on FPGA-based accelerators. Together, these results indicate that Allo enables scalable, composable accelerators that can outperform traditional flows on realistic multi-kernel workloads.

Abstract

Special-purpose hardware accelerators are increasingly pivotal for sustaining performance improvements in emerging applications, especially as the benefits of technology scaling continue to diminish. However, designers currently lack effective tools and methodologies to construct complex, high-performance accelerator architectures in a productive manner. Existing high-level synthesis (HLS) tools often require intrusive source-level changes to attain satisfactory quality of results. Despite the introduction of several new accelerator design languages (ADLs) aiming to enhance or replace HLS, their advantages are more evident in relatively simple applications with a single kernel. Existing ADLs prove less effective for realistic hierarchical designs with multiple kernels, even if the design hierarchy is flattened. In this paper, we introduce Allo, a composable programming model for efficient spatial accelerator design. Allo decouples hardware customizations, including compute, memory, communication, and data type from algorithm specification, and encapsulates them as a set of customization primitives. Allo preserves the hierarchical structure of an input program by combining customizations from different functions in a bottom-up, type-safe manner. This approach facilitates holistic optimizations that span across function boundaries. We conduct comprehensive experiments on commonly-used HLS benchmarks and several realistic deep learning models. Our evaluation shows that Allo can outperform state-of-the-art HLS tools and ADLs on all test cases in the PolyBench. For the GPT2 model, the inference latency of the Allo generated accelerator is 1.7x faster than the NVIDIA A100 GPU with 5.4x higher energy efficiency, demonstrating the capability of Allo to handle large-scale designs.