Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Boolean Logic as an Error feedback mechanism

Louis Leconte

TL;DR

This work tackles the NP-hard problem of training Binary Neural Networks via Boolean Logic backpropagation by introducing a continuous relaxation that embeds the discrete optimization signal into a gradient-like process through quantizers $Q_0$ and $Q_1$. It proves a convergence guarantee under standard non-convex assumptions, yielding a bound on the average gradient norm with an additive quantization error floor $2L d \kappa$, and shows the result extends to unbiased quantization schemes. The analysis relies on constructing a virtual sequence and leveraging error-feedback mechanisms, establishing convergence to first-order stationary points in a smooth non-convex setting. Practically, the results justify theoretical convergence for Boolean-based training on resource-constrained hardware and illuminate the trade-offs imposed by discretization.

Abstract

The notion of Boolean logic backpropagation was introduced to build neural networks with weights and activations being Boolean numbers. Most of computations can be done with Boolean logic instead of real arithmetic, both during training and inference phases. But the underlying discrete optimization problem is NP-hard, and the Boolean logic has no guarantee. In this work we propose the first convergence analysis, under standard non-convex assumptions.

Boolean Logic as an Error feedback mechanism

TL;DR

This work tackles the NP-hard problem of training Binary Neural Networks via Boolean Logic backpropagation by introducing a continuous relaxation that embeds the discrete optimization signal into a gradient-like process through quantizers and . It proves a convergence guarantee under standard non-convex assumptions, yielding a bound on the average gradient norm with an additive quantization error floor , and shows the result extends to unbiased quantization schemes. The analysis relies on constructing a virtual sequence and leveraging error-feedback mechanisms, establishing convergence to first-order stationary points in a smooth non-convex setting. Practically, the results justify theoretical convergence for Boolean-based training on resource-constrained hardware and illuminate the trade-offs imposed by discretization.

Abstract

The notion of Boolean logic backpropagation was introduced to build neural networks with weights and activations being Boolean numbers. Most of computations can be done with Boolean logic instead of real arithmetic, both during training and inference phases. But the underlying discrete optimization problem is NP-hard, and the Boolean logic has no guarantee. In this work we propose the first convergence analysis, under standard non-convex assumptions.
Paper Structure (7 sections, 3 theorems, 25 equations, 3 algorithms)

This paper contains 7 sections, 3 theorems, 25 equations, 3 algorithms.

Table of Contents

  1. Introduction
  2. Continuous Abstraction of nguyen2023boolean
  3. Non-convex analysis
  4. Preliminaries
  5. Proof of \ref{['theorem:cvgquant']}
  6. Conclusion
  7. Acknowledgments

Key Result

Theorem 3.1

Assume assum:uniflowerbound to assum:stoflip. Boolean Logic applied to Boolean weights $w$ converges at rate: where $A^* = {2(f(w_0)-f_*)}$, $B^*= 2L\sigma^2$, $C^*= 4L^2\sigma^2 \frac{\delta}{(1-\delta)^2}$, $r_d = \frac{d\kappa}{2}$.

Theorems & Definitions (10)

  • Theorem 3.1
  • Remark 3.2
  • Remark 3.3
  • Remark 3.4
  • Remark 3.5
  • Lemma 3.6
  • proof
  • Lemma 3.7
  • proof
  • proof