Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Repair Brain Damage: Real-Numbered Error Correction Code for Neural Network

Ziqing Li, Myung Cho, Qiutong Jin, Weiyu Xu

TL;DR

This work tackles neural network robustness to internal faults by introducing a real-numbered error correction code that imposes linear constraints on both layer weights and outputs. It develops a linear programming-based detection and recovery scheme that can identify and correct memory and datapath errors with minimal overhead, employing a projection-based method to enforce constraints without adding trainable parameters. Empirical results on MNIST and CIFAR-10 show the approach corrects a substantial fraction of sparse weight and computation errors (up to around 210 corrupted parameters out of thousands) while preserving baseline accuracy. The proposed method offers a scalable, parameter-efficient path toward reliable DNNs suitable for safety-critical applications and hardware where memory and computation may be unreliable.

Abstract

We consider a neural network (NN) that may experience memory faults and computational errors. In this paper, we propose a novel real-number-based error correction code (ECC) capable of detecting and correcting both memory errors and computational errors. The proposed approach introduces structures in the form of real-number-based linear constraints on the NN weights to enable error detection and correction, without sacrificing classification performance or increasing the number of real-valued NN parameters.

Repair Brain Damage: Real-Numbered Error Correction Code for Neural Network

TL;DR

This work tackles neural network robustness to internal faults by introducing a real-numbered error correction code that imposes linear constraints on both layer weights and outputs. It develops a linear programming-based detection and recovery scheme that can identify and correct memory and datapath errors with minimal overhead, employing a projection-based method to enforce constraints without adding trainable parameters. Empirical results on MNIST and CIFAR-10 show the approach corrects a substantial fraction of sparse weight and computation errors (up to around 210 corrupted parameters out of thousands) while preserving baseline accuracy. The proposed method offers a scalable, parameter-efficient path toward reliable DNNs suitable for safety-critical applications and hardware where memory and computation may be unreliable.

Abstract

We consider a neural network (NN) that may experience memory faults and computational errors. In this paper, we propose a novel real-number-based error correction code (ECC) capable of detecting and correcting both memory errors and computational errors. The proposed approach introduces structures in the form of real-number-based linear constraints on the NN weights to enable error detection and correction, without sacrificing classification performance or increasing the number of real-valued NN parameters.
Paper Structure (10 sections, 7 equations, 3 figures, 1 algorithm)

This paper contains 10 sections, 7 equations, 3 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Related Works
  4. Linear Programming-based Error Correction Code for Error Tolerant DNN
  5. LP-based error detection and correction
  6. Constructing a DNN to enable the linear programming-based error detection and correction
  7. Numerical Experiments
  8. MNIST Dataset
  9. CIFAR-10 Dataset
  10. Conclusion

Figures (3)

  • Figure 1: Numerical results of MNIST test classification accuracy for ECC-DNN with and without error correction under varying numbers of weight errors.
  • Figure 2: Numerical results of the successful recovery percentage as a function of the number of constraint matrices ${\mathbf B}_j$'s.
  • Figure 3: Numerical results of CIFAR-10 test classification accuracy for ECC-DNN, with and without error correction