Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

WARP-LUTs -- Walsh-Assisted Relaxation for Probabilistic Look Up Tables

Lino Gerlach, Liv Våge, Thore Gerlach, Elliott Kauffman, Isobel Ojalvo

TL;DR

The paper introduces WARP-LUTs, a Walsh-assisted relaxation approach to train probabilistic LUTs with far fewer parameters than prior DLGN-based methods, aiming to speed up convergence while maintaining accuracy. It leverages a Walsh–Hadamard transform for compact, differentiable Boolean-function parameterization and employs a sigmoid relaxation plus Gumbel reparameterization to bridge training and inference. CIFAR-10 experiments show faster convergence and substantially reduced parameter counts across large and small architectures, with comparable final accuracy to baselines. The work highlights potential FPGA-friendly deployment and scalability to higher-input LUTs, outlining future work on optimized kernels and broader benchmarking against LUT-based methods.

Abstract

Fast and efficient machine learning is of growing interest to the scientific community and has spurred significant research into novel model architectures and hardware-aware design. Recent hard? and software co-design approaches have demonstrated impressive results with entirely multiplication-free models. Differentiable Logic Gate Networks (DLGNs), for instance, provide a gradient-based framework for learning optimal combinations of low-level logic gates, setting state-of-the-art trade-offs between accuracy, resource usage, and latency. However, these models suffer from high computational cost during training and do not generalize well to logic blocks with more inputs. In this work, we introduce Walsh-Assisted Relaxation for Probabilistic Look-Up Tables (WARP-LUTs) - a novel gradient-based method that efficiently learns combinations of logic gates with substantially fewer trainable parameters. We demonstrate that WARP-LUTs achieve significantly faster convergence on CIFAR-10 compared to DLGNs, while maintaining comparable accuracy. Furthermore, our approach suggests potential for extension to higher-input logic blocks, motivating future research on extremely efficient deployment on modern FPGAs and its real-time science applications.

WARP-LUTs -- Walsh-Assisted Relaxation for Probabilistic Look Up Tables

TL;DR

The paper introduces WARP-LUTs, a Walsh-assisted relaxation approach to train probabilistic LUTs with far fewer parameters than prior DLGN-based methods, aiming to speed up convergence while maintaining accuracy. It leverages a Walsh–Hadamard transform for compact, differentiable Boolean-function parameterization and employs a sigmoid relaxation plus Gumbel reparameterization to bridge training and inference. CIFAR-10 experiments show faster convergence and substantially reduced parameter counts across large and small architectures, with comparable final accuracy to baselines. The work highlights potential FPGA-friendly deployment and scalability to higher-input LUTs, outlining future work on optimized kernels and broader benchmarking against LUT-based methods.

Abstract

Fast and efficient machine learning is of growing interest to the scientific community and has spurred significant research into novel model architectures and hardware-aware design. Recent hard? and software co-design approaches have demonstrated impressive results with entirely multiplication-free models. Differentiable Logic Gate Networks (DLGNs), for instance, provide a gradient-based framework for learning optimal combinations of low-level logic gates, setting state-of-the-art trade-offs between accuracy, resource usage, and latency. However, these models suffer from high computational cost during training and do not generalize well to logic blocks with more inputs. In this work, we introduce Walsh-Assisted Relaxation for Probabilistic Look-Up Tables (WARP-LUTs) - a novel gradient-based method that efficiently learns combinations of logic gates with substantially fewer trainable parameters. We demonstrate that WARP-LUTs achieve significantly faster convergence on CIFAR-10 compared to DLGNs, while maintaining comparable accuracy. Furthermore, our approach suggests potential for extension to higher-input logic blocks, motivating future research on extremely efficient deployment on modern FPGAs and its real-time science applications.
Paper Structure (12 sections, 2 equations, 2 figures, 2 tables)

This paper contains 12 sections, 2 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Example: 2-input logic gates
  5. Relaxation
  6. Reducing Discretization Gap
  7. Experiments
  8. Limitations and Future Work
  9. Acknowledgements
  10. Source Code
  11. WH-Transform-based Decompositions for Two-Input Boolean Functions
  12. Model Architectures

Figures (2)

  • Figure 1: Validation accuracy on CIFAR-10. Logic gate activations are represented as continuous numbers (relaxed) during training and discrete values during inference. Our implementation shows faster convergence compared to the baseline.
  • Figure 2: Results of a very small (35968 gates) convolutional model on CIFAR-10 for DLGN (left) and WARP-LUT (right). We compare the validation accuracy, where the solid line shows the discrete mode and the dashed line shows the relaxed mode (top) along with the distribution of logic gates after training (bottom). Blue depicts random weight initialization, orange residual weight initialization.