Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Evaluating Different Fault Injection Abstractions on the Assessment of DNN SW Hardening Strategies

Giuseppe Esposito, Juan David Guerrero-Balaguera, Josie Esteban Rodriguez Condia, Matteo Sonza Reorda

TL;DR

Problem: how to reliably evaluate SW-based hardening for DNNs under hardware faults. Approach: compare ISA-level HITPT fault injection with APP-level fault injection across three DNNs and three hardening strategies. Key contributions: first direct cross-abstraction study showing ISA FI can invert hardening rankings and reveal DUEs and higher Critical-SDCs, with a BER-aligned fair comparison method. Reliability is quantified by the Relative Accuracy Degradation $RAD = (ACC_{fault-free}-ACC_{faulty})/ACC_{fault-free}$, highlighting the practical impact for safeguarding safety-critical AI deployments.

Abstract

The reliability of Neural Networks has gained significant attention, prompting efforts to develop SW-based hardening techniques for safety-critical scenarios. However, evaluating hardening techniques using application-level fault injection (FI) strategies, which are commonly hardware-agnostic, may yield misleading results. This study for the first time compares two FI approaches (at the application level (APP) and instruction level (ISA)) to evaluate deep neural network SW hardening strategies. Results show that injecting permanent faults at ISA (a more detailed abstraction level than APP) changes completely the ranking of SW hardening techniques, in terms of both reliability and accuracy. These results highlight the relevance of using an adequate analysis abstraction for evaluating such techniques.

Evaluating Different Fault Injection Abstractions on the Assessment of DNN SW Hardening Strategies

TL;DR

Problem: how to reliably evaluate SW-based hardening for DNNs under hardware faults. Approach: compare ISA-level HITPT fault injection with APP-level fault injection across three DNNs and three hardening strategies. Key contributions: first direct cross-abstraction study showing ISA FI can invert hardening rankings and reveal DUEs and higher Critical-SDCs, with a BER-aligned fair comparison method. Reliability is quantified by the Relative Accuracy Degradation , highlighting the practical impact for safeguarding safety-critical AI deployments.

Abstract

The reliability of Neural Networks has gained significant attention, prompting efforts to develop SW-based hardening techniques for safety-critical scenarios. However, evaluating hardening techniques using application-level fault injection (FI) strategies, which are commonly hardware-agnostic, may yield misleading results. This study for the first time compares two FI approaches (at the application level (APP) and instruction level (ISA)) to evaluate deep neural network SW hardening strategies. Results show that injecting permanent faults at ISA (a more detailed abstraction level than APP) changes completely the ranking of SW hardening techniques, in terms of both reliability and accuracy. These results highlight the relevance of using an adequate analysis abstraction for evaluating such techniques.

Paper Structure

This paper contains 10 sections, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related works
  3. Evaluation methodology
  4. Hardware-Injection Through Program Transformation (HITPT)
  5. Application-level FI
  6. Evaluation process
  7. Experimental results
  8. Error distribution impacts from both evaluation abstractions
  9. Impact of faults on DNN accuracy
  10. Conclusions

Figures (2)

  • Figure 1: Permanent fault propagation from the hardware to the application possibly generating an error.
  • Figure 2: Fault distribution for 3 DNNs where FI campaigns target i) weight parameters by performing single bit-flips (Weight SBFs) at the APP, ii) Functional Units (FUs) at ISA and iii) Registers (Regs) at ISA. Specifically, the proportion of faults is indicated in relation to each compromised component. Moreover, each bar corresponds to the baseline (BL) unhardened model, Swap ReLU6 (SR), Ranger (R) and Adaptive Clipper (AC).