Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Impactful Bit-Flip Search on Full-precision Models

Nadav Benedek, Matan Levy, Mahmood Sharif

TL;DR

Bit-flip attacks on full-precision networks can catastrophically degrade performance with only a handful of bit changes. The authors propose Impactful Bit-Flip Search (IBS) that uses gradient-based bit ranking, emphasizing exponent MSB bits, and a Weight-Stealth variant that preserves the original weight distribution; baselines include Random Bit Flip and Exhaustive Search. The results show model-wise and layer-wise attacks achieve high Relative Accuracy Drop ($RAD$) with few flips, approaching Exhaustive Search, while Weight-Stealth scales $RAD$ with flips and can reach near-chance performance while remaining stealthy. This work reveals strong security risks from hardware fault attacks and informs tamper-detection and robust defense strategies for full-precision networks.

Abstract

Neural networks have shown remarkable performance in various tasks, yet they remain susceptible to subtle changes in their input or model parameters. One particularly impactful vulnerability arises through the Bit-Flip Attack (BFA), where flipping a small number of critical bits in a model's parameters can severely degrade its performance. A common technique for inducing bit flips in DRAM is the Row-Hammer attack, which exploits frequent uncached memory accesses to alter data. Identifying susceptible bits can be achieved through exhaustive search or progressive layer-by-layer analysis, especially in quantized networks. In this work, we introduce Impactful Bit-Flip Search (IBS), a novel method for efficiently pinpointing and flipping critical bits in full-precision networks. Additionally, we propose a Weight-Stealth technique that strategically modifies the model's parameters in a way that maintains the float values within the original distribution, thereby bypassing simple range checks often used in tamper detection.

Impactful Bit-Flip Search on Full-precision Models

TL;DR

Bit-flip attacks on full-precision networks can catastrophically degrade performance with only a handful of bit changes. The authors propose Impactful Bit-Flip Search (IBS) that uses gradient-based bit ranking, emphasizing exponent MSB bits, and a Weight-Stealth variant that preserves the original weight distribution; baselines include Random Bit Flip and Exhaustive Search. The results show model-wise and layer-wise attacks achieve high Relative Accuracy Drop () with few flips, approaching Exhaustive Search, while Weight-Stealth scales with flips and can reach near-chance performance while remaining stealthy. This work reveals strong security risks from hardware fault attacks and informs tamper-detection and robust defense strategies for full-precision networks.

Abstract

Neural networks have shown remarkable performance in various tasks, yet they remain susceptible to subtle changes in their input or model parameters. One particularly impactful vulnerability arises through the Bit-Flip Attack (BFA), where flipping a small number of critical bits in a model's parameters can severely degrade its performance. A common technique for inducing bit flips in DRAM is the Row-Hammer attack, which exploits frequent uncached memory accesses to alter data. Identifying susceptible bits can be achieved through exhaustive search or progressive layer-by-layer analysis, especially in quantized networks. In this work, we introduce Impactful Bit-Flip Search (IBS), a novel method for efficiently pinpointing and flipping critical bits in full-precision networks. Additionally, we propose a Weight-Stealth technique that strategically modifies the model's parameters in a way that maintains the float values within the original distribution, thereby bypassing simple range checks often used in tamper detection.

Paper Structure

This paper contains 11 sections, 7 equations, 1 figure, 2 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Technical Approach
  4. Weight-stealth
  5. Method
  6. Baseline methods
  7. Our methods
  8. Experiments
  9. Results & Discussion
  10. Conclusion
  11. Contributions

Figures (1)

  • Figure 1: Post attack accuracy comparison between Weight-Stealth attacks with different number of bit flips per iterations. $n$ denotes the number of bit flips per iteration. The target model is the 50K CNN. Lower is better. Shaded areas are standard deviations.