LogHD: Robust Compression of Hyperdimensional Classifiers via Logarithmic Class-Axis Reduction

TL;DR

LogHD introduces class-axis compression for hyperdimensional computing by replacing $C$ class prototypes with $n\ge\lceil \log_k C\rceil$ bundles and decoding in an $n$-dimensional activation space. This reduces memory from $O(CD)$ to $O(D\log_k C)$ while preserving the full feature dimension $D$, aided by a capacity-aware codebook and activation-profile decoding with optional refinement. The approach yields competitive accuracy and superior robustness to bit-flip and quantization noise, and scales favorably on ASIC hardware with substantial energy and latency advantages over CPU/GPU baselines and feature-axis compression. Hybrid configurations combining LogHD with SparseHD offer tunable memory-accuracy-robustness trade-offs, making LogHD a practical, hardware-friendly strategy for robust memory-centric ML.

Abstract

Hyperdimensional computing (HDC) suits memory, energy, and reliability-constrained systems, yet the standard "one prototype per class" design requires $O(CD)$ memory (with $C$ classes and dimensionality $D$). Prior compaction reduces $D$ (feature axis), improving storage/compute but weakening robustness. We introduce LogHD, a logarithmic class-axis reduction that replaces the $C$ per-class prototypes with $n\!\approx\!\lceil\log_k C\rceil$ bundle hypervectors (alphabet size $k$) and decodes in an $n$-dimensional activation space, cutting memory to $O(D\log_k C)$ while preserving $D$. LogHD uses a capacity-aware codebook and profile-based decoding, and composes with feature-axis sparsification. Across datasets and injected bit flips, LogHD attains competitive accuracy with smaller models and higher resilience at matched memory. Under equal memory, it sustains target accuracy at roughly $2.5$-$3.0\times$ higher bit-flip rates than feature-axis compression; an ASIC instantiation delivers $498\times$ energy efficiency and $62.6\times$ speedup over an AMD Ryzen 9 9950X and $24.3\times$/$6.58\times$ over an NVIDIA RTX 4090, and is $4.06\times$ more energy-efficient and $2.19\times$ faster than a feature-axis HDC ASIC baseline.

Figures (6)

• Figure 1: Comparison of approaches. "Clean performance" denotes accuracy and throughput measured in fault-free conditions (no injected bit-flip noise). (a) Feature-axis compression (SparseHD): reduce $D$ with sparsity $S$; improves storage/compute, but robustness degrades as $D$ shrinks. (b) LogHD (class-axis): keep $D$ and replace $C$ prototypes with $n\!\approx\!\lceil\log_k C\rceil{+}\varepsilon$ bundles ($\varepsilon \geq 0$); memory scales logarithmically in $C$ while maintaining high-$D$ robustness. (c) Hybrid (LogHD + SparseHD): combine class- and feature-axis compression to tune memory, clean performance, and robustness.
• Figure 2: LogHD overview. (a) Initial bundling: assign each class a length-$n$$k$-ary code $B_i$; map symbols via $g$ and form bundles $\{\mathbf{M}_j\}$ by weighted superposition; a minimax-load selector balances per-bundle capacity. (b) Activation profiling: compute activation vectors $A(\mathbf{x})$ against bundles and estimate per-class means $\mathbf{P}_y$. (c) Inference: classify by nearest profile in activation space. (d) Iterative refinement: nudge bundles toward code-implied targets with a perceptron-style update. Replacing $C$ prototypes by $n\!\approx\!\lceil\log_k C\rceil$ bundles reduces memory to $\mathcal{O}(D\log_k C)$ while preserving dimensionality and robustness.
• Figure 3: Accuracy under random bit flips. Test accuracy versus bit-flip probability $p$ at matched model-size budgets $(\leq x)$ across datasets, comparing SparseHD, LogHD ($k\in\{2,3\}$), and Hybrid.
• Figure 4: Sensitivity to dimensionality and quantization. Test accuracy on UCIHAR versus bit-flip probability $p$ for varying hypervector dimensionality $D$ and numeric precision (1, 2, 4, 8 bits) at matched model-size budgets $(\leq x)$.
• Figure 5: Effect of alphabet size $k$. Test accuracy on Page and UCIHAR while varying $n/C$ for different $k$, bit precisions, and flip probabilities $p\in\{0,0.8\}$. For each $k$, the curve sweeps $n$ starting at the feasibility limit $n\ge\lceil\log_k C\rceil$.