Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Power-Efficient Hardware Implementation of L-Mul

Ruiqi Chen, Yangxintong Lyu, Han Bao, Bruno da Silva

TL;DR

This work presents a power-efficient FPGA implementation of the L-Mul approximate FP8 multiplier for neural network inference. It leverages LUT and carry-chain primitives on AMD Xilinx UltraScale/UltraScale+ devices to implement exponent and mantissa adders, with a piecewise L-Mul approximation that reduces multiplications to shifts/adders. The design achieves low resource usage and favorable energy efficiency, outperforming comparable 8-bit designs and enabling DSP-free CNN and GCN accelerators with validated accuracy trade-offs on standard datasets. The results demonstrate practical impact for energy-constrained NN workloads and suggest applicability to larger models such as LLMs and diffusion models.

Abstract

Multiplication is a core operation in modern neural network (NN) computations, contributing significantly to energy consumption. The linear-complexity multiplication (L-Mul) algorithm is specifically proposed as an approximate multiplication method for emerging NN models, such as large language model (LLM), to reduce the energy consumption and computational complexity of multiplications. However, hardware implementation designs for L-Mul have not yet been reported. Additionally, 8-bit floating-point (FP8), as an emerging data format, offers a better dynamic range compared to traditional 8-bit integer (INT8), making it increasingly popular and widely adopted in NN computations. This paper thus presents a power-efficient FPGAbased hardware implementation (approximate FP8 multiplier) for L-Mul. The core computation is implemented using the dynamic reconfigurable lookup tables and carry chains primitives available in AMD Xilinx UltraScale/UltraScale+ technology. The accuracy and resource utilization of the approximate multiplier are evaluated and analyzed. Furthermore, the FP8 approximate multiplier is deployed in the inference phase of representative NN models to validate its effectiveness.

A Power-Efficient Hardware Implementation of L-Mul

TL;DR

This work presents a power-efficient FPGA implementation of the L-Mul approximate FP8 multiplier for neural network inference. It leverages LUT and carry-chain primitives on AMD Xilinx UltraScale/UltraScale+ devices to implement exponent and mantissa adders, with a piecewise L-Mul approximation that reduces multiplications to shifts/adders. The design achieves low resource usage and favorable energy efficiency, outperforming comparable 8-bit designs and enabling DSP-free CNN and GCN accelerators with validated accuracy trade-offs on standard datasets. The results demonstrate practical impact for energy-constrained NN workloads and suggest applicability to larger models such as LLMs and diffusion models.

Abstract

Multiplication is a core operation in modern neural network (NN) computations, contributing significantly to energy consumption. The linear-complexity multiplication (L-Mul) algorithm is specifically proposed as an approximate multiplication method for emerging NN models, such as large language model (LLM), to reduce the energy consumption and computational complexity of multiplications. However, hardware implementation designs for L-Mul have not yet been reported. Additionally, 8-bit floating-point (FP8), as an emerging data format, offers a better dynamic range compared to traditional 8-bit integer (INT8), making it increasingly popular and widely adopted in NN computations. This paper thus presents a power-efficient FPGAbased hardware implementation (approximate FP8 multiplier) for L-Mul. The core computation is implemented using the dynamic reconfigurable lookup tables and carry chains primitives available in AMD Xilinx UltraScale/UltraScale+ technology. The accuracy and resource utilization of the approximate multiplier are evaluated and analyzed. Furthermore, the FP8 approximate multiplier is deployed in the inference phase of representative NN models to validate its effectiveness.

Paper Structure

This paper contains 13 sections, 12 equations, 5 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. FP8 Formats
  4. FPGA Structure
  5. The Hardware Implementation for L-Mul
  6. L-Mul for FP8
  7. Hardware Design
  8. Results and Discussion
  9. Experimental setup
  10. Error Evaluation
  11. Hardware Implementation and Evaluation
  12. Deployment and Evaluation on Computation of DNN Models
  13. Conclusions

Figures (5)

  • Figure 1: The demonstration of the INT8 and FP8 (E4M3) defined in IEEE 754. MSB stands for most significant bit and LSB stands for least significant bit.
  • Figure 2: Typical AMD FPGA configurable logic block (CLB) structure ultra. \ref{['fig_clb']} Logic cell within the CLB. \ref{['fig_lut']} LUT6 structure. \ref{['fig_carry']} Carry chain structure.
  • Figure 3: Hardware fine-grained architecture for L-Mul.
  • Figure 4: The normalized number of unique error occurrences under different FP8 formats.
  • Figure 5: The Pareto analysis under area and latency.