Comprehensive Design Space Exploration for Tensorized Neural Network Hardware Accelerators

TL;DR

This work addresses hardware-aware deployment of tensorized neural networks by introducing a comprehensive design space exploration framework that jointly optimizes contraction paths, core partitioning, and dataflow mappings to minimize end-to-end edge latency. A MAC-guided path pruning strategy and latency-driven global search identify model- and layer-level configurations that maximize hardware efficiency, then are realized on a configurable FPGA with a streaming TT contraction kernel and a parameterizable systolic GEMM engine. Experimental results across CNN and ViT benchmarks show substantial end-to-end speedups (3.28x–4.00x inference, 3.42x–3.85x training) and significant energy savings (up to ~21% power reduction), validating the framework’s ability to bridge algorithmic tensorization with practical hardware performance. The contributions demonstrate that jointly optimizing contraction paths, dataflow, and hardware architecture yields superior deployment efficiency on edge devices compared to MAC-optimized or dense baselines.

Abstract

High-order tensor decomposition has been widely adopted to obtain compact deep neural networks for edge deployment. However, existing studies focus primarily on its algorithmic advantages such as accuracy and compression ratio-while overlooking the hardware deployment efficiency. Such hardware-unaware designs often obscure the potential latency and energy benefits of tensorized models. Although several works attempt to reduce computational cost by optimizing the contraction sequence based on the number of multiply-accumulate operations, they typically neglect the underlying hardware characteristics, resulting in suboptimal real-world performance. We observe that the contraction path, hardware architecture, and dataflow mapping are tightly coupled and must be optimized jointly within a unified design space to maximize deployment efficiency on real devices. To this end, we propose a co-exploration framework that unifies these dimensions within a unified design space for efficient training and inference of tensorized neural networks on edge platforms. The framework formulates a latency oriented search objective and solves it via a global latency-driven exploration across the unified design space to achieve end-to-end model efficiency. The optimized configurations are implemented on a configurable FPGA kernel, achieving up to 4x and 3.85x lower inference and training latency compared with the dense baseline.

Figures (5)

• Figure 1: Tensor graph representations of (a) matrix, (b) tensor, (c) matrix multiplication, (d) tensor contraction, (e) tensor train (TT) format linear layer, and (f) tensor train (TT) format convolution layer.
• Figure 2: Overview of the proposed DSE framework. We first construct the layer-wise design space comprising contraction paths $\mathcal{P}_l$, core partitioning options $\mathcal{C}_l$, and dataflow mappings $\mathcal{D}_l$. Next, we simulate the costs of all feasible configuration combinations for each layer. Finally, a global search identifies the Pareto-optimal $(\mathcal{P}^*, \mathcal{C}^*, \mathcal{D}^*)$ configuration that minimizes the overall model execution cost for efficient edge deployment.
• Figure 3: The MACs and latency of different contraction sequences for a tensorized layer. Left: reconstruction-based contraction sequence; Middle: MAC-optimal contraction sequence; Right: latency-optimal contraction sequence.
• Figure 4: Parameterizable $M_\mathrm{PE}\!\times\!N_\mathrm{PE}$ systolic-style array with shared on-chip memory hierarchy.
• Figure 5: Latency comparison across Top-K contraction paths, different dataflow mappings, and core partitioning.