Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hardware-Aware Neural Network Compilation with Learned Optimization: A RISC-V Accelerator Approach

Ravindra Ganti, Steve Xu

TL;DR

XgenSilicon ML Compiler delivers a fully automated, hardware-aware pipeline that maps ONNX/TF/PyTorch models to optimized RISC-V assembly for custom ASICs. It fuses a learned cost model with a five-algorithm auto-tuning framework, extreme quantization with full KL calibration, and validation-driven compilation to minimize hardware-risk while optimizing performance, power, and area. Dynamic shape support and advanced cache-aware cost modeling enable efficient deployment across variable input shapes and memory hierarchies. Experimental results show 2.5–4.5× speedups, 3–6× power reduction, and 40–60% area reduction versus baselines, with rapid compilation times and hardware-validated outputs. The approach supports a broad set of ONNX operators, RISCV Vector optimizations, and ASIC synthesis-ready code, positioning it as a practical solution for rapid ASIC development cycles.

Abstract

We present XgenSilicon ML Compiler, a fully automated end-to-end compilation framework that transforms high-level machine learning models into optimized RISC-V assembly code for custom ASIC accelerators. By unifying the system's cost model across software and hardware, the compiler achieves significant improvements in Power, Performance, and Area (PPA) metrics compared to standard off-the-shelf components and hand-designed chips through five key innovations: (1) a multi-algorithm auto-tuning framework with five search strategies (Bayesian Optimization, Genetic Algorithm, Simulated Annealing, Random Search, Grid Search) combined with a learned cost model, (2) an integrated quantization framework supporting extreme precisions from FP32 to Binary with full KL divergence calibration (2048-bin histogram optimization) and momentum-based QAT gradient updates, (3) hardware-aware validation ensuring 100 percent ISA compliance and memory constraint satisfaction, (4) dynamic shape support with multi-configuration specialization, and (5) advanced cache-aware cost modeling with multi-level cache hierarchy analysis. Our evaluation demonstrates that ASICs produced by this compiler achieve 2.5-4.5x better performance, 3-6x lower power consumption, and 40-60 percent area reduction compared to baseline implementations. The compiler supports more than 100 ONNX operators across 12 categories, implements advanced RISC-V Vector optimizations, and generates hardware-validated assembly code suitable for direct ASIC synthesis. All compilation steps are fully automated, requiring zero manual intervention from model input to ASIC-ready output.

Hardware-Aware Neural Network Compilation with Learned Optimization: A RISC-V Accelerator Approach

TL;DR

XgenSilicon ML Compiler delivers a fully automated, hardware-aware pipeline that maps ONNX/TF/PyTorch models to optimized RISC-V assembly for custom ASICs. It fuses a learned cost model with a five-algorithm auto-tuning framework, extreme quantization with full KL calibration, and validation-driven compilation to minimize hardware-risk while optimizing performance, power, and area. Dynamic shape support and advanced cache-aware cost modeling enable efficient deployment across variable input shapes and memory hierarchies. Experimental results show 2.5–4.5× speedups, 3–6× power reduction, and 40–60% area reduction versus baselines, with rapid compilation times and hardware-validated outputs. The approach supports a broad set of ONNX operators, RISCV Vector optimizations, and ASIC synthesis-ready code, positioning it as a practical solution for rapid ASIC development cycles.

Abstract

We present XgenSilicon ML Compiler, a fully automated end-to-end compilation framework that transforms high-level machine learning models into optimized RISC-V assembly code for custom ASIC accelerators. By unifying the system's cost model across software and hardware, the compiler achieves significant improvements in Power, Performance, and Area (PPA) metrics compared to standard off-the-shelf components and hand-designed chips through five key innovations: (1) a multi-algorithm auto-tuning framework with five search strategies (Bayesian Optimization, Genetic Algorithm, Simulated Annealing, Random Search, Grid Search) combined with a learned cost model, (2) an integrated quantization framework supporting extreme precisions from FP32 to Binary with full KL divergence calibration (2048-bin histogram optimization) and momentum-based QAT gradient updates, (3) hardware-aware validation ensuring 100 percent ISA compliance and memory constraint satisfaction, (4) dynamic shape support with multi-configuration specialization, and (5) advanced cache-aware cost modeling with multi-level cache hierarchy analysis. Our evaluation demonstrates that ASICs produced by this compiler achieve 2.5-4.5x better performance, 3-6x lower power consumption, and 40-60 percent area reduction compared to baseline implementations. The compiler supports more than 100 ONNX operators across 12 categories, implements advanced RISC-V Vector optimizations, and generates hardware-validated assembly code suitable for direct ASIC synthesis. All compilation steps are fully automated, requiring zero manual intervention from model input to ASIC-ready output.

Paper Structure

This paper contains 42 sections, 13 equations, 7 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. ML Compilation Frameworks
  4. Quantization Techniques
  5. ASIC Compilation
  6. Auto-Tuning Algorithms
  7. Our Approach
  8. Compiler Architecture
  9. System Overview
  10. Learned Optimization Framework
  11. Learned Cost Model
  12. Training Sample Collection
  13. Hybrid Cost Model
  14. Multi-Algorithm Auto-Tuning Framework
  15. Quantization Framework
  16. ...and 27 more sections

Figures (7)

  • Figure 1: Complete compilation pipeline from ONNX model to ASIC-ready RISC-V assembly
  • Figure 2: Performance speedup achieved by XgenSilicon-compiled ASICs
  • Figure 3: Power consumption comparison across platforms
  • Figure 4: Silicon area comparison: XgenSilicon ASIC vs. Hand-designed ASIC
  • Figure 5: Auto-tuning convergence: Learned vs. Analytical cost model
  • ...and 2 more figures