Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Laplace-HDC: Understanding the geometry of binary hyperdimensional computing

Saeid Pourmand, Wyatt D. Whiting, Alireza Aghasi, Nicholas F. Marshall

TL;DR

The paper analyzes binary hyperdimensional computing (HDC) through the geometry of the binding operation and shows that the induced data similarity aligns with a Laplace-like kernel. This leads to Laplace-HDC, a practical binary HDC scheme that achieves higher accuracy by carefully selecting admissible kernels and using structured permutations, yielding a similarity $S(\boldsymbol{x},\boldsymbol{y})=\prod_{i=1}^d \boldsymbol{K}(\boldsymbol{x}(i), \boldsymbol{y}(i))$ with a principled bandwidth. It further develops a family of admissible kernels $\boldsymbol{K}_\alpha$ that approximate Laplace behavior and demonstrates improved performance on MNIST/FashionMNIST, including robustness to bit corruptions and spatially aware encoding via Haar/convolutional features. A translation-equivariant embedding is proposed to recover spatial information in image data, complemented by Haar and CNN-derived features to boost performance. Overall, the work advances binary HDC by linking kernel geometry to encoding design, offering practical encodings and robust, spatially aware variants with competitive accuracy.

Abstract

This paper studies the geometry of binary hyperdimensional computing (HDC), a computational scheme in which data are encoded using high-dimensional binary vectors. We establish a result about the similarity structure induced by the HDC binding operator and show that the Laplace kernel naturally arises in this setting, motivating our new encoding method Laplace-HDC, which improves upon previous methods. We describe how our results indicate limitations of binary HDC in encoding spatial information from images and discuss potential solutions, including using Haar convolutional features and the definition of a translation-equivariant HDC encoding. Several numerical experiments highlighting the improved accuracy of Laplace-HDC in contrast to alternative methods are presented. We also numerically study other aspects of the proposed framework such as robustness and the underlying translation-equivariant encoding.

Laplace-HDC: Understanding the geometry of binary hyperdimensional computing

TL;DR

The paper analyzes binary hyperdimensional computing (HDC) through the geometry of the binding operation and shows that the induced data similarity aligns with a Laplace-like kernel. This leads to Laplace-HDC, a practical binary HDC scheme that achieves higher accuracy by carefully selecting admissible kernels and using structured permutations, yielding a similarity with a principled bandwidth. It further develops a family of admissible kernels that approximate Laplace behavior and demonstrates improved performance on MNIST/FashionMNIST, including robustness to bit corruptions and spatially aware encoding via Haar/convolutional features. A translation-equivariant embedding is proposed to recover spatial information in image data, complemented by Haar and CNN-derived features to boost performance. Overall, the work advances binary HDC by linking kernel geometry to encoding design, offering practical encodings and robust, spatially aware variants with competitive accuracy.

Abstract

This paper studies the geometry of binary hyperdimensional computing (HDC), a computational scheme in which data are encoded using high-dimensional binary vectors. We establish a result about the similarity structure induced by the HDC binding operator and show that the Laplace kernel naturally arises in this setting, motivating our new encoding method Laplace-HDC, which improves upon previous methods. We describe how our results indicate limitations of binary HDC in encoding spatial information from images and discuss potential solutions, including using Haar convolutional features and the definition of a translation-equivariant HDC encoding. Several numerical experiments highlighting the improved accuracy of Laplace-HDC in contrast to alternative methods are presented. We also numerically study other aspects of the proposed framework such as robustness and the underlying translation-equivariant encoding.
Paper Structure (27 sections, 5 theorems, 92 equations, 7 figures, 4 tables)

This paper contains 27 sections, 5 theorems, 92 equations, 7 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Motivation
  3. Preliminaries and Notation
  4. Defining an Embedding
  5. Constructing Hypervectors
  6. Main Analytic Result
  7. Choosing an Admissible Kernel
  8. Heuristic Derivation
  9. Precise Description
  10. Translation Equivariant Encoding
  11. Experiments
  12. Classification Methods
  13. Float and Binary Majority Vote
  14. Float and Binary SGD
  15. Laplace-HDC in its Basic Form
  16. ...and 12 more sections

Key Result

Lemma 1.1

Let $\boldsymbol{g}$ be an $n$-dimensional vector with i.i.d. random standard Gaussian entries. Then, for any fixed vectors $\boldsymbol{u},\boldsymbol{v} \in \mathbb{S}^{n-1}$, we have where $\mathbb{S}^{n-1} = \{ x \in \mathbb{R}^n : \|x\|_2 = 1\}$.

Figures (7)

  • Figure 1: Commutative diagram for translation-equivariance of binding operation \ref{['2dencoding:scheme']}.
  • Figure 2: Collection of $9$ Haar convolution matrices of dimension $4 \times 4$.
  • Figure 3: Accuracy histograms of the methods in Table \ref{['tab:comp']} for 50 trials. The mean accuracy and one-standard deviation interval are shown with dashed lines: (a) Vanilla HDC, (b) Extended HoloGN, (c) OnlineHD, (d) RFF-HDC, (e) Laplace-HDC
  • Figure 4: The robustness of the proposed HDC formulation to noise: in each hypervector, a portion of the bits are randomly flipped, and the accuracy is evaluated for the noisy model. The narrow shaded region around the accuracy curve is $\pm$3 times the standard deviation of the accuracy. One can notice that up to almost 25% bit error rate, the accuracy does not drop below 80%.
  • Figure 5: One example from each class in Fashion MNSIT dataset
  • ...and 2 more figures

Theorems & Definitions (15)

  • Remark 1.1: 1D-Cyclic family
  • Remark 1.2: 1D-Block Cyclic family
  • Lemma 1.1: Grothendieck's identity
  • Theorem 1.1
  • Corollary 1.1: Informal Statement
  • Lemma 1.2
  • proof : Proof of Lemma \ref{['lemschoenberg']}
  • Theorem 1.2
  • Remark 1.3: Laplace kernel
  • Remark 1.4: Limitation of similarity structure
  • ...and 5 more