Thales: Formulating and Estimating Architectural Vulnerability Factors for DNN Accelerators

TL;DR

Thales introduces Resiliency Accuracy (RA), a DNN-specific metric that measures the network’s inference accuracy under transient hardware faults. It reframes RA as an expected value over software fault sites by transferring hardware fault probabilities to software variables (probability transfer) and estimates RA efficiently via Monte Carlo integration with importance sampling guided by DNN-specific heuristics. The authors validate the method against RTL fault-injection data, showing that conventional software fault-injection-based RA overestimates resilience and that using actual utilization factors improves accuracy estimates. They further demonstrate how RA-aware NAS can design DNNs with higher resilience and provide comprehensive comparisons across networks and accelerators, highlighting practical implications for fault-tolerant DNN deployment.

Abstract

As Deep Neural Networks (DNNs) are increasingly deployed in safety critical and privacy sensitive applications such as autonomous driving and biometric authentication, it is critical to understand the fault-tolerance nature of DNNs. Prior work primarily focuses on metrics such as Failures In Time (FIT) rate and the Silent Data Corruption (SDC) rate, which quantify how often a device fails. Instead, this paper focuses on quantifying the DNN accuracy given that a transient error has occurred, which tells us how well a network behaves when a transient error occurs. We call this metric Resiliency Accuracy (RA). We show that existing RA formulation is fundamentally inaccurate, because it incorrectly assumes that software variables (model weights/activations) have equal faulty probability under hardware transient faults. We present an algorithm that captures the faulty probabilities of DNN variables under transient faults and, thus, provides correct RA estimations validated by hardware. To accelerate RA estimation, we reformulate RA calculation as a Monte Carlo integration problem, and solve it using importance sampling driven by DNN specific heuristics. Using our lightweight RA estimation method, we show that transient faults lead to far greater accuracy degradation than what todays DNN resiliency tools estimate. We show how our RA estimation tool can help design more resilient DNNs by integrating it with a Network Architecture Search framework.

Figures (10)

• Figure 1: The probability a transient fault is received by an input activation, a weight, an output activation, or a control variable across layers in LeNet-5 and MNIST-Hogwild. Software variables do not have uniform faulty probabilities. Control variables do not correspond to variables in a DNN model --- they are proxies for modeling control FFs; see Sec. \ref{['sec:robust:def']} for details.
• Figure 2: An (not-to-scale) illustration of the idea of calculating the faulty probability of a software fault site. Each cell represents a hardware fault site (i.e., a particular FF at a particular cycle). Each stripe pattern (color) represents a software variable, which can occupy multiple cells because of data reuse. The goal is to calculate the number of cells each software variable is mapped to.
• Figure 3: Comparison of two PDFs used in sampling. Note that the PDF must integrate to 1 by definition.
• Figure 4: Comparison of ground truth FF fault probabilities ($y$-axis) with analytically derived probabilities by Thales ($x$-axis).
• Figure 5: Convergence comparison of different sampling strategies. PoC stands for Point of Convergence. Overall, the order of convergence speed is IS-B$>$IS$>$MAC$>$Uniform, which does not converge within given several thousands samples and produces physically unrealizable results.