Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Large-Margin Hyperdimensional Computing: A Learning-Theoretical Perspective

Nikita Zeulin, Olga Galinina, Ravikumar Balakrishnan, Nageen Himayat, Sergey Andreev

TL;DR

A maximum-margin HDC classifier is proposed, which significantly outperforms baseline HDC methods on several benchmark datasets and leverages a formal relation between HDC and support vector machines (SVMs) that is established for the first time.

Abstract

Overparameterized machine learning (ML) methods such as neural networks may be prohibitively resource intensive for devices with limited computational capabilities. Hyperdimensional computing (HDC) is an emerging resource efficient and low-complexity ML method that allows hardware efficient implementations of (re-)training and inference procedures. In this paper, we propose a maximum-margin HDC classifier, which significantly outperforms baseline HDC methods on several benchmark datasets. Our method leverages a formal relation between HDC and support vector machines (SVMs) that we established for the first time. Our findings may inspire novel HDC methods with potentially more hardware-oriented implementations compared to SVMs, thus enabling more efficient learning solutions for various intelligent resource-constrained applications.

Large-Margin Hyperdimensional Computing: A Learning-Theoretical Perspective

TL;DR

A maximum-margin HDC classifier is proposed, which significantly outperforms baseline HDC methods on several benchmark datasets and leverages a formal relation between HDC and support vector machines (SVMs) that is established for the first time.

Abstract

Overparameterized machine learning (ML) methods such as neural networks may be prohibitively resource intensive for devices with limited computational capabilities. Hyperdimensional computing (HDC) is an emerging resource efficient and low-complexity ML method that allows hardware efficient implementations of (re-)training and inference procedures. In this paper, we propose a maximum-margin HDC classifier, which significantly outperforms baseline HDC methods on several benchmark datasets. Our method leverages a formal relation between HDC and support vector machines (SVMs) that we established for the first time. Our findings may inspire novel HDC methods with potentially more hardware-oriented implementations compared to SVMs, thus enabling more efficient learning solutions for various intelligent resource-constrained applications.
Paper Structure (26 sections, 27 equations, 2 figures, 3 tables)

This paper contains 26 sections, 27 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Motivation
  3. Related work
  4. Structure-Preserving HD Representations
  5. Hardware-Oriented HDC Research
  6. HDC-Powered ML Frameworks
  7. Relation Between HDC and Classical ML Methods
  8. Our Contribution
  9. Background and Problem Formulation
  10. Binary HDC Classification
  11. Binary Linear $C$-SVM Classification
  12. Relation between HDC and SVM
  13. Primal HDC Optimization Problem
  14. Dual HDC Optimization Problem
  15. Maximum-Margin HDC Classifier
  16. ...and 11 more sections

Figures (2)

  • Figure 1: Comparison of performance of maximum-margin HDC (blue), linear $C$-SVM (orange), OnlineHD (green), perceptron-based HDC (red), and other algorithms.
  • Figure 2: Performance comparison of maximum-margin HDC (blue), linear $C$-SVM (orange), OnlineHD (green), and perceptron-based HDC (red) for MNIST dataset and different sizes of hypervectors $D$.