Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Approximate Multiplier Induced Error Propagation in Deep Neural Networks

A. M. H. H. Alahakoon, Hassaan Saadat, Darshana Jayasinghe, Sri Parameswaran

TL;DR

This paper addresses how approximate multiplier errors propagate through GEMM in deep neural networks and affect inference accuracy. It develops a formal framework that links multiplier error moments (mean μ and variance σ^2) to matrix-level distortion quantified by the Frobenius norm, showing that bias is the principal driver of distortion in realistic DNN dimensions. The authors validate the theory via software-based synthetic error injection and hardware experiments using an error-configurable MBM multiplier on an FPGA-based Gemmini DNN accelerator, demonstrating strong correlation between predicted distortion and actual accuracy across ImageNet-scale networks. The work enables rapid, architecture-agnostic prediction of AxM impact on DNN performance and provides practical guidance for designing accuracy-preserving approximate multipliers. It also outlines a roadmap for automation and extension to other approximation techniques in future accelerator design.

Abstract

Deep Neural Networks (DNNs) rely heavily on dense arithmetic operations, motivating the use of Approximate Multipliers (AxMs) to reduce energy consumption in hardware accelerators. However, a rigorous mathematical characterization of how AxMs error distributions influence DNN accuracy remains underdeveloped. This work presents an analytical framework that connects the statistical error moments of an AxM to the induced distortion in General Matrix Multiplication (GEMM). Using the Frobenius norm of the resulting error matrix, we derive a closed form expression for practical DNN dimensions that demonstrates the distortion is predominantly governed by the multiplier mean error (bias). To evaluate this model in realistic settings, we incorporate controlled error injection into GEMM and convolution layers and examine its effect on ImageNet scale networks. The predicted distortion correlates strongly with the observed accuracy degradation, and an error configurable AxM case study implemented on an FPGA further confirms the analytical trends. By providing a lightweight alternative to behavioral or hardware level simulations, this framework enables rapid estimation of AxM impact on DNN inference quality.

Approximate Multiplier Induced Error Propagation in Deep Neural Networks

TL;DR

This paper addresses how approximate multiplier errors propagate through GEMM in deep neural networks and affect inference accuracy. It develops a formal framework that links multiplier error moments (mean μ and variance σ^2) to matrix-level distortion quantified by the Frobenius norm, showing that bias is the principal driver of distortion in realistic DNN dimensions. The authors validate the theory via software-based synthetic error injection and hardware experiments using an error-configurable MBM multiplier on an FPGA-based Gemmini DNN accelerator, demonstrating strong correlation between predicted distortion and actual accuracy across ImageNet-scale networks. The work enables rapid, architecture-agnostic prediction of AxM impact on DNN performance and provides practical guidance for designing accuracy-preserving approximate multipliers. It also outlines a roadmap for automation and extension to other approximation techniques in future accelerator design.

Abstract

Deep Neural Networks (DNNs) rely heavily on dense arithmetic operations, motivating the use of Approximate Multipliers (AxMs) to reduce energy consumption in hardware accelerators. However, a rigorous mathematical characterization of how AxMs error distributions influence DNN accuracy remains underdeveloped. This work presents an analytical framework that connects the statistical error moments of an AxM to the induced distortion in General Matrix Multiplication (GEMM). Using the Frobenius norm of the resulting error matrix, we derive a closed form expression for practical DNN dimensions that demonstrates the distortion is predominantly governed by the multiplier mean error (bias). To evaluate this model in realistic settings, we incorporate controlled error injection into GEMM and convolution layers and examine its effect on ImageNet scale networks. The predicted distortion correlates strongly with the observed accuracy degradation, and an error configurable AxM case study implemented on an FPGA further confirms the analytical trends. By providing a lightweight alternative to behavioral or hardware level simulations, this framework enables rapid estimation of AxM impact on DNN inference quality.

Paper Structure

This paper contains 17 sections, 22 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background and Related work
  3. GEMM operation: Building block of DNN computations
  4. Approximate Multipliers for DNN computations
  5. Mathematical Error Analysis
  6. Individual Approximate Multiplier Error Model
  7. GEMM error formulation
  8. Frobenius Norm as a Metric for Matrix Shape Deviation
  9. Accumulated Expected Frobenius Norm for DNNs
  10. Experimental Setup
  11. Software based synthetic error injection setup
  12. Hardware based validation setup
  13. Results and Discussion
  14. Error injection and validation
  15. Hardware Evaluation Using an Error configurable approximate multiplier
  16. ...and 2 more sections

Figures (2)

  • Figure 1: Distortion of the resultant matrix in $\mathbb{R}^{3}$ under numerical error
  • Figure 2: Model based variation of accuracy with synthetic numerical error injection. Column a: Top-5 Accuracy. Column b: Accumulated Expected Squared Frobenius Norm. Column c: Inverse Frobenius Norm (Scaled and Capped).