From Arithmetic to Logic: The Resilience of Logic and Lookup-Based Neural Networks Under Parameter Bit-Flips

Abstract

The deployment of deep neural networks (DNNs) in safety-critical edge environments necessitates robustness against hardware-induced bit-flip errors. While empirical studies indicate that reducing numerical precision can improve fault tolerance, the theoretical basis of this phenomenon remains underexplored. In this work, we study resilience as a structural property of neural architectures rather than solely as a property of a dataset-specific trained solution. By deriving the expected squared error (MSE) under independent parameter bit flips across multiple numerical formats and layer primitives, we show that lower precision, higher sparsity, bounded activations, and shallow depth are consistently favored under this corruption model. We then argue that logic and lookup-based neural networks realize the joint limit of these design trends. Through ablation studies on the MLPerf Tiny benchmark suite, we show that the observed empirical trends are consistent with the theoretical predictions, and that LUT-based models remain highly stable in corruption regimes where standard floating-point models fail sharply. Furthermore, we identify a novel even-layer recovery effect unique to logic-based architectures and analyze the structural conditions under which it emerges. Overall, our results suggest that shifting from continuous arithmetic weights to discrete Boolean lookups can provide a favorable accuracy-resilience trade-off for hardware fault tolerance.

Figures (8)

• Figure 1: Illustration of a 2-layer neural network composed entirely of layers of lookup tables with binary values. The outputs of each layer address the LUTs of the next layer, culminating in a popcount for class activations.
• Figure 2: Numerical Precision Ablation. Accuracy vs. Bit Error Rate ($p$) averaged across datasets. A clear hierarchy emerges: floating-point models collapse early ($p \approx 10^{-5}$) due to exponent errors, while integer and binary models sustain accuracy significantly longer.
• Figure 3: Width Ablation. Wider networks (e.g., $W=1024$) degrade faster than narrower ones, validating the $O(n^2)$ bias accumulation hypothesis.
• Figure 4: Depth Ablation. Deeper networks suffer from error avalanches, confirming the multiplicative error propagation in standard architectures.
• Figure 5: Activation Function Ablation. Hard saturation functions (Sign, low-temp Sigmoid) provide superior error masking compared to unbounded ReLU.

Theorems & Definitions (22)

• Theorem 1: Expected Error for Integer Weights
• proof
• Theorem 2: Expected Error for Floating-Point Weights
• proof
• Theorem 3: Layer-wise AQ Error with Protected Quantization Parameters
• proof
• Theorem 4: Structure of Layer-wise AQ Error with Corrupted Quantization Parameters
• proof
• Theorem 5: Expected MSE for BNNs
• proof