HEAL: Brain-inspired Hyperdimensional Efficient Active Learning

TL;DR

HEAL tackles data efficiency in brain-inspired HyperDimensional Computing by introducing a gradient-free active learning framework tailored for HDC classifiers. It combines an efficient HDC ensemble-based uncertainty estimator with prior hypervectors, and a lightweight diversity mechanism based on hypervector memorization to guide batch acquisitions. The approach yields higher AL quality and dramatically faster acquisition than baselines, including Bayesian and diversity-guided methods, and remains robust in the presence of duplicated samples. Practically, HEAL enables fast, low-labor annotation for edge-friendly ML tasks while preserving or improving predictive performance on diverse datasets.

Abstract

Drawing inspiration from the outstanding learning capability of our human brains, Hyperdimensional Computing (HDC) emerges as a novel computing paradigm, and it leverages high-dimensional vector presentation and operations for brain-like lightweight Machine Learning (ML). Practical deployments of HDC have significantly enhanced the learning efficiency compared to current deep ML methods on a broad spectrum of applications. However, boosting the data efficiency of HDC classifiers in supervised learning remains an open question. In this paper, we introduce Hyperdimensional Efficient Active Learning (HEAL), a novel Active Learning (AL) framework tailored for HDC classification. HEAL proactively annotates unlabeled data points via uncertainty and diversity-guided acquisition, leading to a more efficient dataset annotation and lowering labor costs. Unlike conventional AL methods that only support classifiers built upon deep neural networks (DNN), HEAL operates without the need for gradient or probabilistic computations. This allows it to be effortlessly integrated with any existing HDC classifier architecture. The key design of HEAL is a novel approach for uncertainty estimation in HDC classifiers through a lightweight HDC ensemble with prior hypervectors. Additionally, by exploiting hypervectors as prototypes (i.e., compact representations), we develop an extra metric for HEAL to select diverse samples within each batch for annotation. Our evaluation shows that HEAL surpasses a diverse set of baselines in AL quality and achieves notably faster acquisition than many BNN-powered or diversity-guided AL methods, recording 11 times to 40,000 times speedup in acquisition runtime per batch.

Figures (15)

• Figure 1: Outline of active learning in supervised classification.
• Figure 2: Encode to hypervectors via fractional power encoding.
• Figure 3: Histogram of predictive entropy for HDC classification with naive ensembles on (a): in-distribution testing set and (b): OOD testing set.
• Figure 4: Ensemble HDC classification with prior hypervectors.
• Figure 5: Histogram of predictive entropy for ensemble HDC classification with prior hypervectors on (a): in-distribution testing set and (b): OOD testing set.