Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

WARP Logic Neural Networks

Lino Gerlach, Thore Gerlach, Liv Våge, Elliott Kauffman, Isobel Ojalvo

TL;DR

WARP-LNNs introduce a Walsh–Hadamard-based, differentiable, and parameter-efficient framework for learning Boolean functions, addressing the inefficiencies and discretization gaps of prior logic neural networks. By combining stochastic smoothing (Gumbel-Sigmoid), residual initialization, and learnable thresholding, WARP achieves fast convergence and scalable expressivity to higher-arity LUTs. The WH-based parametrization provides a minimal representation with $2^n$ parameters per neuron while remaining fully expressive, and the framework unifies and subsumes previous approaches as special cases. Empirical results show faster training, effective higher-arity convolutional blocks, and strong scaling relative to state-of-the-art n-LUT methods, signaling practical potential for hardware-native, multiplication-free inference.

Abstract

Fast and efficient AI inference is increasingly important, and recent models that directly learn low-level logic operations have achieved state-of-the-art performance. However, existing logic neural networks incur high training costs, introduce redundancy or rely on approximate gradients, which limits scalability. To overcome these limitations, we introduce WAlsh Relaxation for Probabilistic (WARP) logic neural networks -- a novel gradient-based framework that efficiently learns combinations of hardware-native logic blocks. We show that WARP yields the most parameter-efficient representation for exactly learning Boolean functions and that several prior approaches arise as restricted special cases. Training is improved by introducing learnable thresholding and residual initialization, while we bridge the gap between relaxed training and discrete logic inference through stochastic smoothing. Experiments demonstrate faster convergence than state-of-the-art baselines, while scaling effectively to deeper architectures and logic functions with higher input arity.

WARP Logic Neural Networks

TL;DR

WARP-LNNs introduce a Walsh–Hadamard-based, differentiable, and parameter-efficient framework for learning Boolean functions, addressing the inefficiencies and discretization gaps of prior logic neural networks. By combining stochastic smoothing (Gumbel-Sigmoid), residual initialization, and learnable thresholding, WARP achieves fast convergence and scalable expressivity to higher-arity LUTs. The WH-based parametrization provides a minimal representation with parameters per neuron while remaining fully expressive, and the framework unifies and subsumes previous approaches as special cases. Empirical results show faster training, effective higher-arity convolutional blocks, and strong scaling relative to state-of-the-art n-LUT methods, signaling practical potential for hardware-native, multiplication-free inference.

Abstract

Fast and efficient AI inference is increasingly important, and recent models that directly learn low-level logic operations have achieved state-of-the-art performance. However, existing logic neural networks incur high training costs, introduce redundancy or rely on approximate gradients, which limits scalability. To overcome these limitations, we introduce WAlsh Relaxation for Probabilistic (WARP) logic neural networks -- a novel gradient-based framework that efficiently learns combinations of hardware-native logic blocks. We show that WARP yields the most parameter-efficient representation for exactly learning Boolean functions and that several prior approaches arise as restricted special cases. Training is improved by introducing learnable thresholding and residual initialization, while we bridge the gap between relaxed training and discrete logic inference through stochastic smoothing. Experiments demonstrate faster convergence than state-of-the-art baselines, while scaling effectively to deeper architectures and logic functions with higher input arity.
Paper Structure (37 sections, 2 theorems, 30 equations, 9 figures, 4 tables)

This paper contains 37 sections, 2 theorems, 30 equations, 9 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Function Approximation
  4. Linear Combination
  5. Direct Parametrization
  6. Basis Parametrization
  7. Background
  8. Differentiable Logic Gate Networks
  9. Parametrization
  10. Limitations
  11. Differentiable Weightless Neural Networks
  12. Limitations
  13. Light Logic Neural Networks
  14. Limitations
  15. $\mathop{\mathrm{\text{WARP}}}\nolimits$ Logic Neural Networks
  16. ...and 22 more sections

Key Result

Theorem 4.1

The $\mathop{\mathrm{\text{WARP}}}\nolimits$ parametrization is the most parameter-efficient representation of any boolean function and previous approaches are special cases that either introduce redundancy ($\mathop{\mathrm{\text{DLGN}}}\nolimits$), errors through approximation ($\mathop{\mathrm{\t

Figures (9)

  • Figure 1: Overview of proposed $\mathop{\mathrm{\text{WARP}}}\nolimits$ LNNs enabling differentiable learning of Boolean functions. During training, single neurons are continuously relaxed, while being replaced by a logic function to enable fast and efficient inference. (a) We propose learnable thresholding with relaxing input binarization, which leads to improved performance. (b) For parameterizing neurons, we introduce $\mathop{\mathrm{\text{WARP}}}\nolimits$ which is based on the Walsh-Hadamard transform. This leads to exponentially less parameters ($16$ to $4$ for $n=2$) and deployable logic structures are obtained by optimal thresholding. (c)$\mathop{\mathrm{\text{WARP}}}\nolimits$ is maximally parameter-efficient, does not rely on approximate gradients (exact) and is fully expressive, describing the ability to represent any Boolean function (see \ref{['theo:generality']}). While previous methods mostly satisfy only two of these properties (parameter inefficient petersen2022deep, inexact through approximations bacellar2024differentiable or not fully expressive hoang2025kanelandronic2025neuralut), they can be deduced from $\mathop{\mathrm{\text{WARP}}}\nolimits$ as special cases.
  • Figure 2: Our parametrization compared to methods from the literature for $n=1$. The $S$-shaped curve indicates the sigmoid function.
  • Figure 3: Discrete validation accuracy (top) and discretization gap (bottom) for $\mathop{\mathrm{\text{WARP}}}\nolimits$ on CIFAR-10 comparing different parametrization methods and varying LUT sizes, $n=2$ (left), $n=4$ (middle) and $n=6$ (right).
  • Figure 4: Validation accuracy on CIFAR-10 for different binarization methods. The model architecture resembles LogicTreeNet-M, and the $\mathop{\mathrm{\text{DLGN}}}\nolimits$ parametrization according to \ref{['eq:dlgn_param']}. Each learning curve consists of three runs with different random seeds.
  • Figure 5: Validation accuracy on JSC for different binarization methods vs the number of bits for the binarization of each feature. The models' architectures resemble DWN(n=6, sm) with learnable connections in the first layer. Each configuration was run with ten different random seeds.
  • ...and 4 more figures

Theorems & Definitions (5)

  • Theorem 4.1
  • Theorem 4.2
  • proof
  • proof
  • proof