THDC: Training Hyperdimensional Computing Models with Backpropagation

TL;DR

THDC tackles the limitations of traditional hyperdimensional computing—namely ultra-high dimensionality, fixed Item Memory, and heuristic AM training—by enabling end-to-end learning through backpropagation. It replaces static IM with trainable embeddings ($P$ for positions and $V$ for values) and learns the AM with a one-layer Binary Neural Network, guided by a cross-entropy objective and a straight-through estimator. Evaluated on MNIST, Fashion-MNIST, and CIFAR-10, THDC achieves competitive accuracy at much lower dimensionality, e.g., $D=64$ with strong low-dimensional performance and notable gains over baseline HDC, while reducing the need for $D$ as high as $10{,}000$. This approach advances edge-appropriate HDC by delivering end-to-end trainability and compact representations, narrowing the gap to conventional neural networks.

Abstract

Hyperdimensional computing (HDC) offers lightweight learning for energy-constrained devices by encoding data into high-dimensional vectors. However, its reliance on ultra-high dimensionality and static, randomly initialized hypervectors limits memory efficiency and learning capacity. Therefore, we propose Trainable Hyperdimensional Computing (THDC), which enables end-to-end HDC via backpropagation. THDC replaces randomly initialized vectors with trainable embeddings and introduces a one-layer binary neural network to optimize class representations. Evaluated on MNIST, Fashion-MNIST and CIFAR-10, THDC achieves equal or better accuracy than state-of-the-art HDC, with dimensionality reduced from 10.000 to 64.

Figures (3)

• Figure 1: Color encoding scheme used in THDC.
• Figure 2: Accuracies for different hypervector dimensions on image datasets.
• Figure 3: t-SNE plots comparing baseline HDC (up) and THDC (down) for $D=1000$ on MNIST (left), Fashion-MNIST (middle), and CIFAR-10 (right).