NeuraLUT-Assemble: Hardware-aware Assembling of Sub-Neural Networks for Efficient LUT Inference

TL;DR

NeuraLUT-Assemble tackles the accuracy bottleneck of LUT-based neural networks on FPGAs by assembling multiple small L-LUT neurons into tree-structured, hardware-aware sub-networks. The method interleaves mixed-precision training, learned input mappings, and skip-connections inside LUTs to enable deeper connectivity without exploding LUT input fan-in. It features a full toolflow—from quantization-aware training in PyTorch to RTL generation and FPGA synthesis—that supports configurable pipelining for latency or throughput optimization. Empirical results on MNIST, jet substructure, and network intrusion datasets show substantial area-delay product reductions while maintaining competitive accuracy, outperforming prior LUT-based approaches in efficiency and scalability. The work highlights the potential of hardware-aware AI designs for ultra-low-latency edge inference and provides an open-source framework for broader adoption.

Abstract

Efficient neural networks (NNs) leveraging lookup tables (LUTs) have demonstrated significant potential for emerging AI applications, particularly when deployed on field-programmable gate arrays (FPGAs) for edge computing. These architectures promise ultra-low latency and reduced resource utilization, broadening neural network adoption in fields such as particle physics. However, existing LUT-based designs suffer from accuracy degradation due to the large fan-in required by neurons being limited by the exponential scaling of LUT resources with input width. In practice, in prior work this tension has resulted in the reliance on extremely sparse models. We present NeuraLUT-Assemble, a novel framework that addresses these limitations by combining mixed-precision techniques with the assembly of larger neurons from smaller units, thereby increasing connectivity while keeping the number of inputs of any given LUT manageable. Additionally, we introduce skip-connections across entire LUT structures to improve gradient flow. NeuraLUT-Assemble closes the accuracy gap between LUT-based methods and (fully-connected) MLP-based models, achieving competitive accuracy on tasks such as network intrusion detection, digit classification, and jet classification, demonstrating up to $8.42\times$ reduction in the area-delay product compared to the state-of-the-art at the time of the publication.

Figures (5)

• Figure 1: View of a NeuraLUT L-LUT on the left and a NeuraLUT-Assemble L-LUT tree on the right. $\bullet$ represents an affine transformation, whereas $f$ is the activation function.
• Figure 2: Example of two different NeuraLUT-Assemble configurations for a $16$-input tree. The red connections are decided upon an initial stage of dense training, whereas the black connections always stay fixed.
• Figure 3: Visualization of NeuraLUT-Assemble's toolflow.
• Figure 4: High-level view of toy $6$-layer NeuraLUT-Assemble network.
• Figure 5: Comparison of area (green bar plot) and test accuracy (red box plot) for three architecture options on the JSC dataset. The options are as follows: (1) a $16$-input tree using $4$-LUTs with $2$-depth (corresponding to Option 1 in Figure \ref{['fig:conf']}), (2) a $16$-input tree using 2-LUTs with 4-depth (corresponding to Option 2 in Figure \ref{['fig:conf']}), and (3) a $64$-input tree using $2$-LUTs with $6$-depth (obtained by extending Option 2 with two additional layers). Each architecture was evaluated under three configurations: Complete Model (NeuraLUT-Assemble), without Learned Mappings (using random connectivity), and without Tree-Level Skip-Connections (retaining only intra-L-LUT connections). These configurations introduce negligible variability in area, as the architecture size remains fixed.