Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing LUT-based Deep Neural Networks Inference through Architecture and Connectivity Optimization

Binglei Lou, Ruilin Wu, Philip Leong

TL;DR

SparseLUT tackles two core LUT-DNN challenges on FPGAs: exponential LUT growth with input width and suboptimal random connectivity. It introduces an architecture-level adder that aggregates $A$ PolyLUT sub-neurons to raise effective fan-in to $AF$ and a non-greedy dynamic sparsification training scheme that prunes and regrows connections under a target fan-in $F_o$ without extra hardware overhead. Empirically, SparseLUT yields 2.0x–13.9x LUT reductions and 1.2x–1.6x latency reductions, with accuracy gains up to 2.13% on MNIST and 0.94% on Jet Substructure compared to prior LUT-DNNs. The approach is complemented by a full toolflow—from QAT-aware training to LUT generation and RTL synthesis—and validated across MNIST, Jet Substructure Classification, and CIFAR-10 benchmarks, highlighting practical gains for FPGA-based LUT-DNN deployment.

Abstract

Deploying deep neural networks (DNNs) on resource-constrained edge devices such as FPGAs requires a careful balance among latency, power, and hardware resource usage, while maintaining high accuracy. Existing Lookup Table (LUT)-based DNNs -- such as LogicNets, PolyLUT, and NeuraLUT -- face two critical challenges: the exponential growth of LUT size and inefficient random sparse connectivity. This paper presents SparseLUT, a comprehensive framework that addresses these challenges through two orthogonal optimizations. First, we propose an architectural enhancement that aggregates multiple PolyLUT sub-neurons via an adder, significantly reducing LUT consumption by 2.0x-13.9x and lowering inference latency by 1.2x-1.6x, all while maintaining comparable accuracy. Building upon this foundation, we further introduce a non-greedy training algorithm that optimizes neuron connectivity by selectively pruning less significant inputs and strategically regrowing more effective ones. This training optimization, which incurs no additional area and latency overhead, delivers consistent accuracy improvements across benchmarks -- achieving up to a 2.13% gain on MNIST and 0.94% on Jet Substructure Classification compared to existing LUT-DNN approaches.

Enhancing LUT-based Deep Neural Networks Inference through Architecture and Connectivity Optimization

TL;DR

SparseLUT tackles two core LUT-DNN challenges on FPGAs: exponential LUT growth with input width and suboptimal random connectivity. It introduces an architecture-level adder that aggregates PolyLUT sub-neurons to raise effective fan-in to and a non-greedy dynamic sparsification training scheme that prunes and regrows connections under a target fan-in without extra hardware overhead. Empirically, SparseLUT yields 2.0x–13.9x LUT reductions and 1.2x–1.6x latency reductions, with accuracy gains up to 2.13% on MNIST and 0.94% on Jet Substructure compared to prior LUT-DNNs. The approach is complemented by a full toolflow—from QAT-aware training to LUT generation and RTL synthesis—and validated across MNIST, Jet Substructure Classification, and CIFAR-10 benchmarks, highlighting practical gains for FPGA-based LUT-DNN deployment.

Abstract

Deploying deep neural networks (DNNs) on resource-constrained edge devices such as FPGAs requires a careful balance among latency, power, and hardware resource usage, while maintaining high accuracy. Existing Lookup Table (LUT)-based DNNs -- such as LogicNets, PolyLUT, and NeuraLUT -- face two critical challenges: the exponential growth of LUT size and inefficient random sparse connectivity. This paper presents SparseLUT, a comprehensive framework that addresses these challenges through two orthogonal optimizations. First, we propose an architectural enhancement that aggregates multiple PolyLUT sub-neurons via an adder, significantly reducing LUT consumption by 2.0x-13.9x and lowering inference latency by 1.2x-1.6x, all while maintaining comparable accuracy. Building upon this foundation, we further introduce a non-greedy training algorithm that optimizes neuron connectivity by selectively pruning less significant inputs and strategically regrowing more effective ones. This training optimization, which incurs no additional area and latency overhead, delivers consistent accuracy improvements across benchmarks -- achieving up to a 2.13% gain on MNIST and 0.94% on Jet Substructure Classification compared to existing LUT-DNN approaches.
Paper Structure (22 sections, 2 equations, 10 figures, 9 tables, 2 algorithms)

This paper contains 22 sections, 2 equations, 10 figures, 9 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. LUT-based traditional DNN methods
  4. Sparsification
  5. Methodology
  6. DNN Architecture Optimization
  7. Dynamic Sparsification
  8. Model Representation
  9. Sparse Training
  10. ToolFlow
  11. Results
  12. Datasets
  13. Architectural Optimization Evaluation
  14. PolyLUT-Add vs. original PolyLUT
  15. PolyLUT-Add vs. Deeper and Wider PolyLUT
  16. ...and 7 more sections

Figures (10)

  • Figure 1: Neuron computation with fan-in $N$. LUT-DNNs typically select a random subset (size $F \ll N$) of the inputs. SparseLUT is a training scheme to select the inputs while maximizing accuracy.
  • Figure 2: Optimization strategies of SparseLUT
  • Figure 3: Example iteration showing SparseLUT adjusting neuron connections to reach a target fan-in of 2, introducing new ones to increase the fan-in from 1 to 2 and eliminating connections to reduce the fan-in from 3 to 2. Note that the $\epsilon_2$ parameter is typically set to a much smaller value, requiring multiple iterations to deactivate a connection.
  • Figure 4: Illustration of the generalized LUT-DNN architectures with sparse connectivity.
  • Figure 5: A single-layer block diagram of the optimized architecture.
  • ...and 5 more figures