Hardware-Aware Static Optimization of Hyperdimensional Computations

TL;DR

The paper addresses the challenge of configuring hyperdimensional (HD) computations, specifically binary spatter code (BSC) HD, on error-prone emerging hardware where dynamic tuning is slow and may overfit. It introduces Heim, a static analysis framework that, given a hardware error model and a target accuracy, derives the minimum hypervector size and distance thresholds to guarantee convergence to the target accuracy on expectation. Heim provides a rigorous accuracy analysis for threshold-based and winner-take-all queries, along with an optimization algorithm and a dynamic independence checker, achieving substantial space and time savings while maintaining iso-accuracy across workloads. Comprehensive evaluation on 25 data structures shows Heim delivering 99%+ median accuracy with 1.15x–7.14x smaller hypervectors and 30x–100kx faster parametrization than dynamic tuning, and reveals how emerging technologies such as analog CAMs and ReRAM can be exploited under Heim’s guarantees. The work enables principled, hardware-aware optimization of HD computations, facilitating reliable deployment on noisy, next-generation hardware while providing insights into the benefits and trade-offs of new memory technologies.

Abstract

Binary spatter code (BSC)-based hyperdimensional computing (HDC) is a highly error-resilient approximate computational paradigm suited for error-prone, emerging hardware platforms. In BSC HDC, the basic datatype is a hypervector, a typically large binary vector, where the size of the hypervector has a significant impact on the fidelity and resource usage of the computation. Typically, the hypervector size is dynamically tuned to deliver the desired accuracy; this process is time-consuming and often produces hypervector sizes that lack accuracy guarantees and produce poor results when reused for very similar workloads. We present Heim, a hardware-aware static analysis and optimization framework for BSC HD computations. Heim analytically derives the minimum hypervector size that minimizes resource usage and meets the target accuracy requirement. Heim guarantees the optimized computation converges to the user-provided accuracy target on expectation, even in the presence of hardware error. Heim deploys a novel static analysis procedure that unifies theoretical results from the neuroscience community to systematically optimize HD computations. We evaluate Heim against dynamic tuning-based optimization on 25 benchmark data structures. Given a 99% accuracy requirement, Heim-optimized computations achieve a 99.2%-100.0% median accuracy, up to 49.5% higher than dynamic tuning-based optimization, while achieving 1.15x-7.14x reductions in hypervector size compared to HD computations that achieve comparable query accuracy and finding parametrizations 30.0x-100167.4x faster than dynamic tuning-based approaches. We also use Heim to systematically evaluate the performance benefits of using analog CAMs and multiple-bit-per-cell ReRAM over conventional hardware, while maintaining iso-accuracy -- for both emerging technologies, we find usages where the emerging hardware imparts significant benefits.

Figures (10)

• Figure 1: Illustrative Example: Knowledge Graph
• Figure 2: Dynamic tuning algorithm and Heim specifications.
• Figure 3: Program grammars - Heim accuracy specification language ($Spec$) and hardware model ($Mdl$).
• Figure 4: Visualization of WTA/threshold query over match/no-match distance distributions. Points map to sampled match and no-match query-item memory row distances (• and •) for a 10-element item memory, where match/no-match distances are sampled from match/no-match distance distributions ($\blacksquare$ and $\blacksquare$). Circled points ($\circ$) map to correct row matches for query.
• Figure 5: Heim accuracy analysis