Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Framework to Enable Algorithmic Design Choice Exploration in DNNs

Timothy L. Cronin, Sanmukh Kuppannagari

TL;DR

The paper tackles the problem of identifying and exploiting optimal algorithm choices for DNN operations by introducing ai3, an open-source framework that provides fine-grained algorithmic control and seamless integration with PyTorch. ai3 offers accelerated C++ implementations of core operations, a Python frontend, and support for user-implemented algorithms, enabling two main workflows: swapping the entire backend or swapping specific module types. The framework supports automated and customizable algorithm selection (single-name, per-module lists, or selector functions) and relies on a symbolic trace to construct equivalent DNNs when needed, all with minimal overhead. Experimental evaluations on diverse CNNs show that ai3 can achieve low-latency performance comparable to or better than PyTorch in many configurations, highlighting the practical impact of algorithmic exploration for DNN design and deployment.

Abstract

Deep learning technologies, particularly deep neural networks (DNNs), have demonstrated significant success across many domains. This success has been accompanied by substantial advancements and innovations in the algorithms behind the operations required by DNNs. These enhanced algorithms hold the potential to greatly increase the performance of DNNs. However, discovering the best performing algorithm for a DNN and altering the DNN to use such algorithm is a difficult and time consuming task. To address this, we introduce an open source framework which provides easy to use fine grain algorithmic control for DNNs, enabling algorithmic exploration and selection. Along with built-in high performance implementations of common deep learning operations, the framework enables users to implement and select their own algorithms to be utilized by the DNN. The framework's built-in accelerated implementations are shown to yield outputs equivalent to and exhibit similar performance as implementations in PyTorch, a popular DNN framework. Moreover, the framework incurs no additional performance overhead, meaning that performance depends solely on the algorithms chosen by the user.

A Framework to Enable Algorithmic Design Choice Exploration in DNNs

TL;DR

The paper tackles the problem of identifying and exploiting optimal algorithm choices for DNN operations by introducing ai3, an open-source framework that provides fine-grained algorithmic control and seamless integration with PyTorch. ai3 offers accelerated C++ implementations of core operations, a Python frontend, and support for user-implemented algorithms, enabling two main workflows: swapping the entire backend or swapping specific module types. The framework supports automated and customizable algorithm selection (single-name, per-module lists, or selector functions) and relies on a symbolic trace to construct equivalent DNNs when needed, all with minimal overhead. Experimental evaluations on diverse CNNs show that ai3 can achieve low-latency performance comparable to or better than PyTorch in many configurations, highlighting the practical impact of algorithmic exploration for DNN design and deployment.

Abstract

Deep learning technologies, particularly deep neural networks (DNNs), have demonstrated significant success across many domains. This success has been accompanied by substantial advancements and innovations in the algorithms behind the operations required by DNNs. These enhanced algorithms hold the potential to greatly increase the performance of DNNs. However, discovering the best performing algorithm for a DNN and altering the DNN to use such algorithm is a difficult and time consuming task. To address this, we introduce an open source framework which provides easy to use fine grain algorithmic control for DNNs, enabling algorithmic exploration and selection. Along with built-in high performance implementations of common deep learning operations, the framework enables users to implement and select their own algorithms to be utilized by the DNN. The framework's built-in accelerated implementations are shown to yield outputs equivalent to and exhibit similar performance as implementations in PyTorch, a popular DNN framework. Moreover, the framework incurs no additional performance overhead, meaning that performance depends solely on the algorithms chosen by the user.

Paper Structure

This paper contains 16 sections, 4 figures.

Table of Contents

  1. Introduction
  2. Background
  3. Deep Neural Networks
  4. Algorithmic Design Choices
  5. Relevant Related Works
  6. Framework Overview
  7. Use Cases
  8. DNN Application Development
  9. DNN Algorithm Development
  10. Framework Implementation
  11. Experimental Evaluations
  12. Setup
  13. Algorithm Implementation
  14. Results
  15. Limitations and Future Directions
  16. ...and 1 more sections

Figures (4)

  • Figure 1: Example paths of a PyTorch module through the framework
  • Figure 2: Latency of Algorithms Using Swapped Backend
  • Figure 3: Latency of Algorithms Using Swapped Modules
  • Figure 4: End-to-End Latency of CNNs with Swapped Modules Relative to PyTorch