Analysis of LLM Vulnerability to GPU Soft Errors: An Instruction-Level Fault Injection Study

TL;DR

This work targets the reliability of large language models running on GPUs by performing an instruction-level fault-injection study with NVBitFI. It introduces an end-to-end LLM Reliability Evaluation Framework, analyzes five vulnerability dimensions, and defines the Model Vulnerability Factor ($MVF$) as the sum of $DUE$ and $SDC$ to quantify user-visible failures. Key findings show that fault vulnerability is highly dependent on task, architecture, model scale, bit-position, and operator type; a practical MVF approximation from Instruction Vulnerability Factors accurately tracks actual vulnerability across models. The results offer actionable guidance for fault-tolerance strategies, such as prioritizing protection for high-risk bits and the output layer, while leveraging normalization and architecture-aware design to improve robustness in GPU-accelerated LLM inference.

Abstract

Large language models (LLMs) are highly compute- and memory-intensive, posing significant demands on high-performance GPUs. At the same time, advances in GPU technology driven by shrinking transistor sizes and lower operating voltages have made these devices increasingly susceptible to soft errors. While prior work has examined GPU reliability, most studies have focused on general-purpose applications or conventional neural networks mostly used for vision tasks such as classification and detection. In contrast, systematic analysis of modern large-scale LLMs remains limited, despite their rapid adoption in diverse application scenarios. Given the unique characteristics of LLMs, their resilience to soft errors may differ substantially from earlier models. To bridge this gap, we conduct the first instruction-level fault injection study of LLM inference. Our approach reveals reliability characteristics from multiple perspectives, highlighting the effects of model architecture, parameter scale, and task complexity. These findings provide new insights into LLM reliability and inform the design of more effective fault tolerance mechanisms.

Figures (13)

• Figure 1: Typical fault injection approaches at different abstraction levels (1) Algorithm-level fault injection, it injects bit-flip errors to weights and activations at neural network layers. (2) Instruction-level fault injection, it injects bit-flip errors to GPU instructions. (3) Micro-architecture-level fault injection, it injects bit-flip errors at major registers and components of GPU. Essentially, micro-architecture-level fault injection depicts how hardware errors propagate to instructions and algorithms eventually. It is accurate but can be extremely slow. In contrast, algorithm-level fault injection simplifies the error modeling to network operations and ignores low-level details for higher fault simulation speed. Instruction-level fault injection stays between micro-architecture level and algorithm-level fault injection for balanced fault simulation accuracy and speed.
• Figure 2: The Proposed LLM Reliability Evaluation Framework based on NVBitFI.
• Figure 3: GPU behavior distribution of LLMs inference system under Multi-Fault scenarios.
• Figure 4: Model Vulnerability Factors (MVF) of different LLMs. The error bars represent the standard deviation of the MVF over multiple experiments.
• Figure 5: E1-E8 represent the following types of fault. E1: Out Of Range Address, E2: MMU Fault, E3: Out Of Range Register, E4: Misaligned Address, E5: Illegal Instruction Parameter, E6: PCIe Bus Error, E7: temperature above threshold, E8: others.