Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bespoke Approximation of Multiplication-Accumulation and Activation Targeting Printed Multilayer Perceptrons

Florentia Afentaki, Gurol Saglam, Argyris Kokkinis, Kostas Siozios, Georgios Zervakis, Mehdi B Tahoori

TL;DR

An automated framework for designing ultra-low power Multilayer Perceptron (MLP) classifiers is proposed which employs, for the first time, a holistic approach to approximate all functions of the MLP's neurons: multiplication, accumulation, and activation.

Abstract

Printed Electronics (PE) feature distinct and remarkable characteristics that make them a prominent technology for achieving true ubiquitous computing. This is particularly relevant in application domains that require conformal and ultra-low cost solutions, which have experienced limited penetration of computing until now. Unlike silicon-based technologies, PE offer unparalleled features such as non-recurring engineering costs, ultra-low manufacturing cost, and on-demand fabrication of conformal, flexible, non-toxic, and stretchable hardware. However, PE face certain limitations due to their large feature sizes, that impede the realization of complex circuits, such as machine learning classifiers. In this work, we address these limitations by leveraging the principles of Approximate Computing and Bespoke (fully-customized) design. We propose an automated framework for designing ultra-low power Multilayer Perceptron (MLP) classifiers which employs, for the first time, a holistic approach to approximate all functions of the MLP's neurons: multiplication, accumulation, and activation. Through comprehensive evaluation across various MLPs of varying size, our framework demonstrates the ability to enable battery-powered operation of even the most intricate MLP architecture examined, significantly surpassing the current state of the art.

Bespoke Approximation of Multiplication-Accumulation and Activation Targeting Printed Multilayer Perceptrons

TL;DR

An automated framework for designing ultra-low power Multilayer Perceptron (MLP) classifiers is proposed which employs, for the first time, a holistic approach to approximate all functions of the MLP's neurons: multiplication, accumulation, and activation.

Abstract

Printed Electronics (PE) feature distinct and remarkable characteristics that make them a prominent technology for achieving true ubiquitous computing. This is particularly relevant in application domains that require conformal and ultra-low cost solutions, which have experienced limited penetration of computing until now. Unlike silicon-based technologies, PE offer unparalleled features such as non-recurring engineering costs, ultra-low manufacturing cost, and on-demand fabrication of conformal, flexible, non-toxic, and stretchable hardware. However, PE face certain limitations due to their large feature sizes, that impede the realization of complex circuits, such as machine learning classifiers. In this work, we address these limitations by leveraging the principles of Approximate Computing and Bespoke (fully-customized) design. We propose an automated framework for designing ultra-low power Multilayer Perceptron (MLP) classifiers which employs, for the first time, a holistic approach to approximate all functions of the MLP's neurons: multiplication, accumulation, and activation. Through comprehensive evaluation across various MLPs of varying size, our framework demonstrates the ability to enable battery-powered operation of even the most intricate MLP architecture examined, significantly surpassing the current state of the art.
Paper Structure (20 sections, 3 equations, 5 figures, 5 tables)

This paper contains 20 sections, 3 equations, 5 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Preamble
  3. Background on Printed Electronics
  4. Related Work
  5. Proposed Framework
  6. Preliminaries
  7. Multiplication Approximation
  8. Activation Approximation
  9. QRelu
  10. Approximate Argmax
  11. Accumulation Approximation
  12. Genetic Optimization
  13. High-level Accuracy Evaluation
  14. High-level Area Estimation
  15. Holistic Approximation Flow
  16. ...and 5 more sections

Figures (5)

  • Figure 1: Overview of our proposed framework.
  • Figure 2: Showcase of the impact of power-of-2 weights on a bespoke MAC circuit. On the left bespoke multipliers and a generic adder tree are used. With power-of-2 weights, only a simpler and narrower adder tree is required.
  • Figure 3: Example of our implemented accumulation approximation.
  • Figure 4: Evaluation of the effectiveness of our accumulation approximation. Area is normalized w.r.t. the corresponding QAT-only approximate MLP.
  • Figure 5: (a) Area and (b) power gains of the MLPs generated by our framework compared to state-of-the-art Armeniakos:TCAD2023:cross, Armeniakos:TC2023:codesign and Weller:2021:printed_stoch. All the MLPs feature a $5\%$ accuracy loss from our baseline Mubarik:MICRO:2020:printedml. Values are normalized w.r.t. Mubarik:MICRO:2020:printedml. Y-axis is in logarithmic scale.