Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MEMHD: Memory-Efficient Multi-Centroid Hyperdimensional Computing for Fully-Utilized In-Memory Computing Architectures

Do Yeong Kang, Yeong Hwan Oh, Chanwook Hwang, Jinhee Kim, Kang Eun Jeon, Jong Hwan Ko

TL;DR

MEMHD tackles the challenge of deploying Hyperdimensional Computing on In-Memory Computing platforms by introducing a memory-efficient multi-centroid associative memory with clustering-based initialization and quantization-aware iterative learning. The approach aligns hypervector dimensions with IMC array constraints, enabling full array utilization and one-shot (or few-shot) associative search, while maintaining or improving accuracy relative to larger-dimension baselines. Key contributions include a clustering-driven AM initialization that spreads centroids effectively across columns, 1-bit AM quantization, and an iterative learning process that accounts for quantization, resulting in up to 13.69% accuracy gains at the same memory usage or 13.25× memory efficiency at equal accuracy, and substantial reductions in computation cycles and array usage. Collectively, MEMHD enables efficient, scalable HDC mappings on IMC hardware, with significant energy and performance benefits for resource-constrained deployments.

Abstract

The implementation of Hyperdimensional Computing (HDC) on In-Memory Computing (IMC) architectures faces significant challenges due to the mismatch between highdimensional vectors and IMC array sizes, leading to inefficient memory utilization and increased computation cycles. This paper presents MEMHD, a Memory-Efficient Multi-centroid HDC framework designed to address these challenges. MEMHD introduces a clustering-based initialization method and quantization aware iterative learning for multi-centroid associative memory. Through these approaches and its overall architecture, MEMHD achieves a significant reduction in memory requirements while maintaining or improving classification accuracy. Our approach achieves full utilization of IMC arrays and enables one-shot (or few-shot) associative search. Experimental results demonstrate that MEMHD outperforms state-of-the-art binary HDC models, achieving up to 13.69% higher accuracy with the same memory usage, or 13.25x more memory efficiency at the same accuracy level. Moreover, MEMHD reduces computation cycles by up to 80x and array usage by up to 71x compared to baseline IMC mapping methods when mapped to 128x128 IMC arrays, while significantly improving energy and computation cycle efficiency.

MEMHD: Memory-Efficient Multi-Centroid Hyperdimensional Computing for Fully-Utilized In-Memory Computing Architectures

TL;DR

MEMHD tackles the challenge of deploying Hyperdimensional Computing on In-Memory Computing platforms by introducing a memory-efficient multi-centroid associative memory with clustering-based initialization and quantization-aware iterative learning. The approach aligns hypervector dimensions with IMC array constraints, enabling full array utilization and one-shot (or few-shot) associative search, while maintaining or improving accuracy relative to larger-dimension baselines. Key contributions include a clustering-driven AM initialization that spreads centroids effectively across columns, 1-bit AM quantization, and an iterative learning process that accounts for quantization, resulting in up to 13.69% accuracy gains at the same memory usage or 13.25× memory efficiency at equal accuracy, and substantial reductions in computation cycles and array usage. Collectively, MEMHD enables efficient, scalable HDC mappings on IMC hardware, with significant energy and performance benefits for resource-constrained deployments.

Abstract

The implementation of Hyperdimensional Computing (HDC) on In-Memory Computing (IMC) architectures faces significant challenges due to the mismatch between highdimensional vectors and IMC array sizes, leading to inefficient memory utilization and increased computation cycles. This paper presents MEMHD, a Memory-Efficient Multi-centroid HDC framework designed to address these challenges. MEMHD introduces a clustering-based initialization method and quantization aware iterative learning for multi-centroid associative memory. Through these approaches and its overall architecture, MEMHD achieves a significant reduction in memory requirements while maintaining or improving classification accuracy. Our approach achieves full utilization of IMC arrays and enables one-shot (or few-shot) associative search. Experimental results demonstrate that MEMHD outperforms state-of-the-art binary HDC models, achieving up to 13.69% higher accuracy with the same memory usage, or 13.25x more memory efficiency at the same accuracy level. Moreover, MEMHD reduces computation cycles by up to 80x and array usage by up to 71x compared to baseline IMC mapping methods when mapped to 128x128 IMC arrays, while significantly improving energy and computation cycle efficiency.

Paper Structure

This paper contains 19 sections, 6 equations, 7 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background : HDC Classification
  3. Overall HDC Structure
  4. Hypervector Encoding
  5. Associative Memory Training
  6. Associative Search
  7. MEMHD: Memory-Efficient Multi-Centroid HD
  8. Multi-Centroid AM Initialization
  9. AM Quantization
  10. Quantization-Aware Iterative Learning
  11. In-Memory Inference
  12. EVALUATION
  13. Experimental Setup
  14. Accuracy and Memory Requirement
  15. Accuracy Heatmap : Dimensions and Memory Columns
  16. ...and 4 more sections

Figures (7)

  • Figure 1: Overview of MEMHD
  • Figure 2: Overall MEMHD framework
  • Figure 3: Accuracy and memory requirement (KB). MEMHD used for (a) MNIST (b) FMNIST: 64x64 to 1024x1024 square sizes ($D$x$C$), (c) ISOLET: fixed 128 columns, varied dimensions. We set dimensions of baslines from 256D to 10240D.
  • Figure 4: MEMHD Accuracy Heatmap from 64x64 to 1024x1024.
  • Figure 5: Accuracy comparison between clustering and random sampling.
  • ...and 2 more figures