MicroHD: An Accuracy-Driven Optimization of Hyperdimensional Computing Algorithms for TinyML systems

TL;DR

The paper tackles the challenge of deploying Hyperdimensional Computing (HDC) in TinyML on resource-constrained edge devices. It introduces MicroHD, an accuracy-driven optimization that co-optimizes multiple HDC hyper-parameters using binary-search and greedy strategies, with offline retraining to enforce user-defined accuracy. MicroHD achieves substantial resource reductions—up to $266\times$ in memory and compute for accuracy losses under $1\%$ and up to several-fold gains relative to state-of-the-art approaches—while remaining applicable to both ID-level and non-linear projection encodings. The method also demonstrates practical benefits for processing-in-memory (PIM) accelerators and federated learning settings, improving energy efficiency and reducing cloud data transfer. The work provides a scalable, offline optimization framework with log-linear complexity, enabling robust, accuracy-controlled compression of HDC workloads for real-world TinyML deployments.

Abstract

Hyperdimensional computing (HDC) is emerging as a promising AI approach that can effectively target TinyML applications thanks to its lightweight computing and memory requirements. Previous works on HDC showed that limiting the standard 10k dimensions of the hyperdimensional space to much lower values is possible, reducing even more HDC resource requirements. Similarly, other studies demonstrated that binary values can be used as elements of the generated hypervectors, leading to significant efficiency gains at the cost of some degree of accuracy degradation. Nevertheless, current optimization attempts do not concurrently co-optimize HDC hyper-parameters, and accuracy degradation is not directly controlled, resulting in sub-optimal HDC models providing several applications with unacceptable output qualities. In this work, we propose MicroHD, a novel accuracy-driven HDC optimization approach that iteratively tunes HDC hyper-parameters, reducing memory and computing requirements while ensuring user-defined accuracy levels. The proposed method can be applied to HDC implementations using different encoding functions, demonstrates good scalability for larger HDC workloads, and achieves compression and efficiency gains up to 200x when compared to baseline implementations for accuracy degradations lower than 1%.

Figures (4)

• Figure 1: Basics of HDC (left). The encoding stage can be implemented using different $\phi$ functions, such as ID-level or projection methods (right).
• Figure 2: MicroHD optimization strategy. An initial analysis determines the impact of different HDC hyper-parameters on memory and computing requirements. Then, a greedy optimization step is applied and the model is re-trained. Finally, an accuracy check controls accuracy to user-defined levels and the procedure repeats until some tunable hyper-parameters exist.
• Figure 3: Achieved compression and workload reduction factors in HDC models using ID-Level and non-linear projection encoding evaluated on multiple datasets imposing a 0.5% (grey), 1.0% (blue), and 5.0% (green) accuracy threshold.
• Figure 4: MicroHD optimized average performance gain factor over the considered benchmarks for accuracy thresholds of 0.5% (grey), 1.0% (blue), and 5.0% (green).