Few-Shot Testing: Estimating Uncertainty of Memristive Deep Neural Networks Using One Bayesian Test Vector

TL;DR

This work tackles uncertainty in memristor-based CIM implementations of deep neural networks by introducing a Bayesian test vector that estimates model uncertainty without retraining or data access. The framework uses a Gaussian variational distribution to optimize a test input $\,\hat{x}$ via ELBO-based objectives, stores only the per-element mean and variance on hardware, and samples efficiently to assess output uncertainty. Across diverse models, tasks, and fault/variation scenarios, the method achieves near 100% coverage with memory overhead as low as $0.024$ MB and minimal forward passes, enabling practical deployment on edge devices. The approach supports pre-deployment and online uncertainty reduction, offers verifiable coverage, and outperforms existing point-estimate and Bayesian-test strategies in both accuracy and efficiency, significantly boosting reliability for memristive CIM-based inference.

Abstract

The performance of deep learning algorithms such as neural networks (NNs) has increased tremendously recently, and they can achieve state-of-the-art performance in many domains. However, due to memory and computation resource constraints, implementing NNs on edge devices is a challenging task. Therefore, hardware accelerators such as computation-in-memory (CIM) with memristive devices have been developed to accelerate the most common operations, i.e., matrix-vector multiplication. However, due to inherent device properties, external environmental factors such as temperature, and an immature fabrication process, memristors suffer from various non-idealities, including defects and variations occurring during manufacturing and runtime. Consequently, there is a lack of complete confidence in the predictions made by the model. To improve confidence in NN predictions made by hardware accelerators in the presence of device non-idealities, in this paper, we propose a Bayesian test vector generation framework that can estimate the model uncertainty of NNs implemented on memristor-based CIM hardware. Compared to the conventional point estimate test vector generation method, our method is more generalizable across different model dimensions and requires storing only one test Bayesian vector in the hardware. Our method is evaluated on different model dimensions, tasks, fault rates, and variation noise to show that it can consistently achieve $100\%$ coverage with only $0.024$ MB of memory overhead.

Figures (8)

• Figure 1: a) Variations of MTJ devices due to manufacturing and thermal variations, b) variations of four MTJs ($P_1\cdots P_4$) in the P-state. The respective variations of the devices are randomly sampled from their distribution, and c) variations in resistance of a single RRAM device in LRS state with different programming cycles $c_1\cdots c_4$.
• Figure 2: Crossbar array of CIM architecture. Each crosspoint consists of a memristor with a 1TIM configuration.
• Figure 3: Distribution of logits on the fault- and variation-free RepVGG ding2021repvgg model on the CIFAR-10 dataset when several randomly sampled inputs from training and validation are applied. The distribution logits change from one input to another.
• Figure 4: Change in output distribution of logits of RepVGG ding2021repvgg model on the CIFAR-10 dataset due to a) variations and b) faults. The spread among logits increases as the noise scale of variations and fault rate increases.
• Figure 5: Relative sensitivity of uncertainty estimates given proposed Bayesian test vector input as well as randomly sampled training and validation. The change in uncertainty estimates is much higher for our proposed Bayesian test vector.