Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Introducing Instruction-Accurate Simulators for Performance Estimation of Autotuning Workloads

Rebecca Pelke, Nils Bosbach, Lennart M. Reimann, Rainer Leupers

TL;DR

This work tackles autotuning for ML workloads under large design spaces by shifting evaluation from real hardware to simulators. It introduces a simulator interface that feeds TVM workloads to instruction-accurate simulators and trains per-architecture, per-kernel score predictors to rank implementations using simulation-derived statistics. Four predictor families (MLR, DNN, Bayesian optimization, XGBoost) are evaluated across x86, ARM, and RISC-V, with results showing the best predictor achieving within the top 3% of true run-time order and sometimes beating native embedded execution when parallelized. The approach enables scalable autotuning when hardware is scarce and lays groundwork for pre-silicon performance estimation, with future work expanding predictor generalization to more CPUs.

Abstract

Accelerating Machine Learning (ML) workloads requires efficient methods due to their large optimization space. Autotuning has emerged as an effective approach for systematically evaluating variations of implementations. Traditionally, autotuning requires the workloads to be executed on the target hardware (HW). We present an interface that allows executing autotuning workloads on simulators. This approach offers high scalability when the availability of the target HW is limited, as many simulations can be run in parallel on any accessible HW. Additionally, we evaluate the feasibility of using fast instruction-accurate simulators for autotuning. We train various predictors to forecast the performance of ML workload implementations on the target HW based on simulation statistics. Our results demonstrate that the tuned predictors are highly effective. The best workload implementation in terms of actual run time on the target HW is always within the top 3 % of predictions for the tested x86, ARM, and RISC-V-based architectures. In the best case, this approach outperforms native execution on the target HW for embedded architectures when running as few as three samples on three simulators in parallel.

Introducing Instruction-Accurate Simulators for Performance Estimation of Autotuning Workloads

TL;DR

This work tackles autotuning for ML workloads under large design spaces by shifting evaluation from real hardware to simulators. It introduces a simulator interface that feeds TVM workloads to instruction-accurate simulators and trains per-architecture, per-kernel score predictors to rank implementations using simulation-derived statistics. Four predictor families (MLR, DNN, Bayesian optimization, XGBoost) are evaluated across x86, ARM, and RISC-V, with results showing the best predictor achieving within the top 3% of true run-time order and sometimes beating native embedded execution when parallelized. The approach enables scalable autotuning when hardware is scarce and lays groundwork for pre-silicon performance estimation, with future work expanding predictor generalization to more CPUs.

Abstract

Accelerating Machine Learning (ML) workloads requires efficient methods due to their large optimization space. Autotuning has emerged as an effective approach for systematically evaluating variations of implementations. Traditionally, autotuning requires the workloads to be executed on the target hardware (HW). We present an interface that allows executing autotuning workloads on simulators. This approach offers high scalability when the availability of the target HW is limited, as many simulations can be run in parallel on any accessible HW. Additionally, we evaluate the feasibility of using fast instruction-accurate simulators for autotuning. We train various predictors to forecast the performance of ML workload implementations on the target HW based on simulation statistics. Our results demonstrate that the tuned predictors are highly effective. The best workload implementation in terms of actual run time on the target HW is always within the top 3 % of predictions for the tested x86, ARM, and RISC-V-based architectures. In the best case, this approach outperforms native execution on the target HW for embedded architectures when running as few as three samples on three simulators in parallel.
Paper Structure (20 sections, 7 equations, 5 figures, 5 tables)

This paper contains 20 sections, 7 equations, 5 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Autotuning in TVM
  4. Cache Hierarchy
  5. The gem5 Simulator
  6. Implementation
  7. Simulator Interface
  8. gem5 Simulator Setup
  9. Score Predictor - Workflow and Notations
  10. Score Predictor Training
  11. Multiple Linear Regression
  12. Regression with DNNs
  13. Bayesian Optimization
  14. XGBoost
  15. Score Predictor Inference
  16. ...and 5 more sections

Figures (5)

  • Figure 1: The proposed simulator interface I and the score predictor approach using instruction-accurate simulators II
  • Figure 2: Autotuning using Auto-Scheduler and AutoTVM
  • Figure 3: Typical cache hierarchies of modern cpu
  • Figure 4: Workflow of training I and execution II of a predictor for one target architecture and one kernel type
  • Figure 5: Sorted run time predictions for the test set of group 3 (a)-(c) when group 3 is included in the training vs. the same samples of group 3 (d)-(f) when group 3 is not included in the training