Lifelong Intelligence Beyond the Edge using Hyperdimensional Computing

TL;DR

LifeHD addresses the need for on-device lifelong learning in edge IoT under streaming, largely unlabeled data. It leverages Hyperdimensional Computing to encode inputs as high-dimensional hypervectors and maintains a two-tier memory of cluster HVs, employing novelty detection, online updates, and spectral clustering-based merging to avoid forgetting. Two extensions, LifeHD$_\textrm{semi}$ and LifeHD$_\textrm{a}$, enable leveraging scarce labels and adaptive dimension pruning for power-constrained deployments. Evaluations on Raspberry Pi and Jetson platforms across MHEALTH, ESC-50, and CIFAR-100 show LifeHD achieving up to $74.8\%$ unsupervised clustering accuracy improvements and up to $34.3\times$ energy efficiency over state-of-the-art NN-based baselines, demonstrating practical viability for autonomous, real-time edge learning. The public code release further supports adoption and reproducibility.

Abstract

On-device learning has emerged as a prevailing trend that avoids the slow response time and costly communication of cloud-based learning. The ability to learn continuously and indefinitely in a changing environment, and with resource constraints, is critical for real sensor deployments. However, existing designs are inadequate for practical scenarios with (i) streaming data input, (ii) lack of supervision and (iii) limited on-board resources. In this paper, we design and deploy the first on-device lifelong learning system called LifeHD for general IoT applications with limited supervision. LifeHD is designed based on a novel neurally-inspired and lightweight learning paradigm called Hyperdimensional Computing (HDC). We utilize a two-tier associative memory organization to intelligently store and manage high-dimensional, low-precision vectors, which represent the historical patterns as cluster centroids. We additionally propose two variants of LifeHD to cope with scarce labeled inputs and power constraints. We implement LifeHD on off-the-shelf edge platforms and perform extensive evaluations across three scenarios. Our measurements show that LifeHD improves the unsupervised clustering accuracy by up to 74.8% compared to the state-of-the-art NN-based unsupervised lifelong learning baselines with as much as 34.3x better energy efficiency. Our code is available at https://github.com/Orienfish/LifeHD.

Figures (15)

• Figure 1: Real-world example of on-device lifelong learning evaluated using the unsupervised clustering accuracy metric xie2016unsupervised. The training latency is measured on two typical edge platforms.
• Figure 2: Spatiotemporal HDC encoding for time-series data. Left: random generation of level hypervectors. Right: the complete encoding process.
• Figure 3: An overview of supervised HDC pipeline including encoding, training and inference (similarity check for classification). An additional pretrained NN is used as a feature extractor in the HDnn.
• Figure 4: The end-to-end algorithm flow of LifeHD.
• Figure 5: An intuitive visualization of cluster HV merging.