Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

NeuraLUT: Hiding Neural Network Density in Boolean Synthesizable Functions

Marta Andronic, George A. Constantinides

TL;DR

NeuraLUT tackles ultra-low-latency FPGA inference by hiding dense, full-precision sub-networks inside logical L-LUTs. By constraining inter-LUT inputs with a fixed fan-in $F$ and bit-width $eta$ and enriching each LUT with skip-connected sub-networks, it achieves higher expressivity with shallower circuit-level depth. A QAT-enabled toolflow converts sub-networks into L-LUTs, generates RTL ROMs, and synthesizes onto FPGA hardware, delivering up to $4.3\times$ latency reduction at fixed accuracy on MNIST and jet substructure tagging, and even larger area-delay improvements relative to prior LUT-based approaches. The work demonstrates significant practical impact for edge FPGA deployments and points to future NAS-driven automatic topology optimization to further exploit NeuraLUT's LUT-centric paradigm.

Abstract

Field-Programmable Gate Array (FPGA) accelerators have proven successful in handling latency- and resource-critical deep neural network (DNN) inference tasks. Among the most computationally intensive operations in a neural network (NN) is the dot product between the feature and weight vectors. Thus, some previous FPGA acceleration works have proposed mapping neurons with quantized inputs and outputs directly to lookup tables (LUTs) for hardware implementation. In these works, the boundaries of the neurons coincide with the boundaries of the LUTs. We propose relaxing these boundaries and mapping entire sub-networks to a single LUT. As the sub-networks are absorbed within the LUT, the NN topology and precision within a partition do not affect the size of the lookup tables generated. Therefore, we utilize fully connected layers with floating-point precision inside each partition, which benefit from being universal function approximators, but with rigid sparsity and quantization enforced between partitions, where the NN topology becomes exposed to the circuit topology. Although cheap to implement, this approach can lead to very deep NNs, and so to tackle challenges like vanishing gradients, we also introduce skip connections inside the partitions. The resulting methodology can be seen as training DNNs with a specific FPGA hardware-inspired sparsity pattern that allows them to be mapped to much shallower circuit-level networks, thereby significantly improving latency. We validate our proposed method on a known latency-critical task, jet substructure tagging, and on the classical computer vision task, digit classification using MNIST. Our approach allows for greater function expressivity within the LUTs compared to existing work, leading to up to $4.3\times$ lower latency NNs for the same accuracy.

NeuraLUT: Hiding Neural Network Density in Boolean Synthesizable Functions

TL;DR

NeuraLUT tackles ultra-low-latency FPGA inference by hiding dense, full-precision sub-networks inside logical L-LUTs. By constraining inter-LUT inputs with a fixed fan-in and bit-width and enriching each LUT with skip-connected sub-networks, it achieves higher expressivity with shallower circuit-level depth. A QAT-enabled toolflow converts sub-networks into L-LUTs, generates RTL ROMs, and synthesizes onto FPGA hardware, delivering up to latency reduction at fixed accuracy on MNIST and jet substructure tagging, and even larger area-delay improvements relative to prior LUT-based approaches. The work demonstrates significant practical impact for edge FPGA deployments and points to future NAS-driven automatic topology optimization to further exploit NeuraLUT's LUT-centric paradigm.

Abstract

Field-Programmable Gate Array (FPGA) accelerators have proven successful in handling latency- and resource-critical deep neural network (DNN) inference tasks. Among the most computationally intensive operations in a neural network (NN) is the dot product between the feature and weight vectors. Thus, some previous FPGA acceleration works have proposed mapping neurons with quantized inputs and outputs directly to lookup tables (LUTs) for hardware implementation. In these works, the boundaries of the neurons coincide with the boundaries of the LUTs. We propose relaxing these boundaries and mapping entire sub-networks to a single LUT. As the sub-networks are absorbed within the LUT, the NN topology and precision within a partition do not affect the size of the lookup tables generated. Therefore, we utilize fully connected layers with floating-point precision inside each partition, which benefit from being universal function approximators, but with rigid sparsity and quantization enforced between partitions, where the NN topology becomes exposed to the circuit topology. Although cheap to implement, this approach can lead to very deep NNs, and so to tackle challenges like vanishing gradients, we also introduce skip connections inside the partitions. The resulting methodology can be seen as training DNNs with a specific FPGA hardware-inspired sparsity pattern that allows them to be mapped to much shallower circuit-level networks, thereby significantly improving latency. We validate our proposed method on a known latency-critical task, jet substructure tagging, and on the classical computer vision task, digit classification using MNIST. Our approach allows for greater function expressivity within the LUTs compared to existing work, leading to up to lower latency NNs for the same accuracy.
Paper Structure (24 sections, 7 equations, 7 figures, 3 tables)

This paper contains 24 sections, 7 equations, 7 figures, 3 tables.

Table of Contents

  1. Introduction and Motivation
  2. Background
  3. DSP-based Neural Networks
  4. XNOR-based Neural Networks
  5. LUT-based Neural Networks
  6. Network in Network
  7. Methodology
  8. Tackling LUT size
  9. Skip-connections
  10. Function hidden inside the L-LUT
  11. Expressive power
  12. Toolflow
  13. Quantization-aware training (QAT)
  14. Sub-network to L-LUT conversion
  15. RTL file generation
  16. ...and 9 more sections

Figures (7)

  • Figure 1: Gray circle: Affine transformation + ReLU. Purple circle: Affine transformation for residual connections. Blue lines: Low precision. Black lines: Full precision.
  • Figure 2: High-level view of NeuraLUT's architecture. Sparsely connected dense sub-networks with skip-connections.
  • Figure 3: Visualization of decision boundaries. Classifier comparison across three different seeds.
  • Figure 4: Visualization of NeuraLUT's toolflow, consisting of four stages.
  • Figure 5: Ablation study on MNIST across 10 seeds. Blue: baseline, Gray: NeuraLUT without skip-connections, Purple: standard NeuraLUT. All models have a fixed circuit-level architecture with ($256,100,100,100,100,10$) L-LUTs.
  • ...and 2 more figures