SafeCiM: Investigating Resilience of Hybrid Floating-Point Compute-in-Memory Deep Learning Accelerators
Swastik Bhattacharya, Sanjay Das, Anand Menon, Shamik Kundu, Arnab Raha, Kanad Basu
TL;DR
The paper tackles the reliability of digital FP Compute-in-Memory (FP-CiM) accelerators for deep learning, introducing a scalable fault-injection framework (FaultCiM) to study bit-flip faults across FP-CiM datapaths. It reveals stage-specific fault sensitivities, with normalization and global alignment being the most vulnerable, and shows that deeper or sparser models can exhibit dramatically different fault responses. Based on these insights, the authors propose SafeCiM, a fault-resilient FP-CiM design that employs post-alignment, an ic×oc×H×W stencil, and a two-stage local/global alignment, achieving up to 49× reduction in accuracy loss from a single adder fault for a 4096-MAC configuration. These results provide actionable architectural guidelines for robust FP-CiM accelerators and highlight the importance of model- and architecture-aware fault tolerance in GenAI edge devices.
Abstract
Deep Neural Networks (DNNs) continue to grow in complexity with Large Language Models (LLMs) incorporating vast numbers of parameters. Handling these parameters efficiently in traditional accelerators is limited by data-transmission bottlenecks, motivating Compute-in-Memory (CiM) architectures that integrate computation within or near memory to reduce data movement. Recent work has explored CiM designs using Floating-Point (FP) and Integer (INT) operations. FP computations typically deliver higher output quality due to their wider dynamic range and precision, benefiting precision-sensitive Generative AI applications. These include models such as LLMs, thus driving advancements in FP-CiM accelerators. However, the vulnerability of FP-CiM to hardware faults remains underexplored, posing a major reliability concern in mission-critical settings. To address this gap, we systematically analyze hardware fault effects in FP-CiM by introducing bit-flip faults at key computational stages, including digital multipliers, CiM memory cells, and digital adder trees. Experiments with Convolutional Neural Networks (CNNs) such as AlexNet and state-of-the-art LLMs including LLaMA-3.2-1B and Qwen-0.3B-Base reveal how faults at each stage affect inference accuracy. Notably, a single adder fault can reduce LLM accuracy to 0%. Based on these insights, we propose a fault-resilient design, SafeCiM, that mitigates fault impact far better than a naive FP-CiM with a pre-alignment stage. For example, with 4096 MAC units, SafeCiM reduces accuracy degradation by up to 49x for a single adder fault compared to the baseline FP-CiM architecture.