CiMNet: Towards Joint Optimization for DNN Architecture and Configuration for Compute-In-Memory Hardware

TL;DR

The proposed CiMNet is a framework that jointly searches for optimal sub-networks and hardware configurations for CiM architectures creating a Pareto optimal frontier of downstream task accuracy and execution metrics (e.g., latency) and can comprehend the complex interplay between a sub-network's performance and the CiM hardware configuration choices including bandwidth, processing element size, and memory size.

Abstract

With the recent growth in demand for large-scale deep neural networks, compute in-memory (CiM) has come up as a prominent solution to alleviate bandwidth and on-chip interconnect bottlenecks that constrain Von-Neuman architectures. However, the construction of CiM hardware poses a challenge as any specific memory hierarchy in terms of cache sizes and memory bandwidth at different interfaces may not be ideally matched to any neural network's attributes such as tensor dimension and arithmetic intensity, thus leading to suboptimal and under-performing systems. Despite the success of neural architecture search (NAS) techniques in yielding efficient sub-networks for a given hardware metric budget (e.g., DNN execution time or latency), it assumes the hardware configuration to be frozen, often yielding sub-optimal sub-networks for a given budget. In this paper, we present CiMNet, a framework that jointly searches for optimal sub-networks and hardware configurations for CiM architectures creating a Pareto optimal frontier of downstream task accuracy and execution metrics (e.g., latency). The proposed framework can comprehend the complex interplay between a sub-network's performance and the CiM hardware configuration choices including bandwidth, processing element size, and memory size. Exhaustive experiments on different model architectures from both CNN and Transformer families demonstrate the efficacy of the CiMNet in finding co-optimized sub-networks and CiM hardware configurations. Specifically, for similar ImageNet classification accuracy as baseline ViT-B, optimizing only the model architecture increases performance (or reduces workload execution time) by 1.7x while optimizing for both the model architecture and hardware configuration increases it by 3.1x.

Figures (4)

• Figure 1: (a) Illustration of the proposed joint architecture-hardware configuration search framework. (b) Illustration of different possible positions for the placement of a dedicated and specialized compute unit to perform MAC operations.
• Figure 2: Illustration of the compute in-memory architecture used in our experiments and evaluation of joint-search.
• Figure 3: MAPE of predictors performing cycle count prediction versus the number of training examples for sub-networks derived from the MobileNetV3, ResNet-50 and ViT-B super-networks (top row). Correlation and Kendall $\tau$ coefficient between actual and predicted values after training the predictor with 1000 examples (bottom row). Green lines show The ideal correlation.
• Figure 4: Pareto fronts in the accuracy / cycle count space of sub-networks optimized from MobileNetV3, ResNet-50 and ViT-B super-networks for both static (blue) and elastic (green) hardware configurations. The performance when using the static, canonical network architecture along with the static hardware configuration is shown by the black square.