Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

NeuroHash: A Hyperdimensional Neuro-Symbolic Framework for Spatially-Aware Image Hashing and Retrieval

Sanggeon Yun, Ryozo Masukawa, SungHeon Jeong, Mohsen Imani

TL;DR

NeuroHash tackles spatially-aware image retrieval in large datasets by integrating pre-trained vision backbones with a Hyperdimensional Computing (HDC) framework to encode spatial structure into high-dimensional vectors. It introduces a self-supervised context-aware HDC encoder, a spatial encoding scheme using positional hypervectors, and a trainable multilinear hyperplane hashing to map high-dimensional representations to compact binary hashes. The system supports conditional retrieval by reweighting global and local symbols, enabling focus on specific objects or regions. Experiments on CIFAR-10 and MS COCO demonstrate competitive mean average precision at 5K ($\mathrm{mAP@5K}$) and a spatial-aware metric $\mathrm{mAP@5K_r}$, with ablations confirming the contribution of all components. The approach offers a flexible neuro-symbolic framework with transparency and potential extensions to temporal information and other modalities.

Abstract

Customizable image retrieval from large datasets remains a critical challenge, particularly when preserving spatial relationships within images. Traditional hashing methods, primarily based on deep learning, often fail to capture spatial information adequately and lack transparency. In this paper, we introduce NeuroHash, a novel neuro-symbolic framework leveraging Hyperdimensional Computing (HDC) to enable highly customizable, spatially-aware image retrieval. NeuroHash combines pre-trained deep neural network models with HDC-based symbolic models, allowing for flexible manipulation of hash values to support conditional image retrieval. Our method includes a self-supervised context-aware HDC encoder and novel loss terms for optimizing lower-dimensional bipolar hashing using multilinear hyperplanes. We evaluate NeuroHash on two benchmark datasets, demonstrating superior performance compared to state-of-the-art hashing methods, as measured by mAP@5K scores and our newly introduced metric, mAP@5Kr, which assesses spatial alignment. The results highlight NeuroHash's ability to achieve competitive performance while offering significant advantages in flexibility and customization, paving the way for more advanced and versatile image retrieval systems.

NeuroHash: A Hyperdimensional Neuro-Symbolic Framework for Spatially-Aware Image Hashing and Retrieval

TL;DR

NeuroHash tackles spatially-aware image retrieval in large datasets by integrating pre-trained vision backbones with a Hyperdimensional Computing (HDC) framework to encode spatial structure into high-dimensional vectors. It introduces a self-supervised context-aware HDC encoder, a spatial encoding scheme using positional hypervectors, and a trainable multilinear hyperplane hashing to map high-dimensional representations to compact binary hashes. The system supports conditional retrieval by reweighting global and local symbols, enabling focus on specific objects or regions. Experiments on CIFAR-10 and MS COCO demonstrate competitive mean average precision at 5K () and a spatial-aware metric , with ablations confirming the contribution of all components. The approach offers a flexible neuro-symbolic framework with transparency and potential extensions to temporal information and other modalities.

Abstract

Customizable image retrieval from large datasets remains a critical challenge, particularly when preserving spatial relationships within images. Traditional hashing methods, primarily based on deep learning, often fail to capture spatial information adequately and lack transparency. In this paper, we introduce NeuroHash, a novel neuro-symbolic framework leveraging Hyperdimensional Computing (HDC) to enable highly customizable, spatially-aware image retrieval. NeuroHash combines pre-trained deep neural network models with HDC-based symbolic models, allowing for flexible manipulation of hash values to support conditional image retrieval. Our method includes a self-supervised context-aware HDC encoder and novel loss terms for optimizing lower-dimensional bipolar hashing using multilinear hyperplanes. We evaluate NeuroHash on two benchmark datasets, demonstrating superior performance compared to state-of-the-art hashing methods, as measured by mAP@5K scores and our newly introduced metric, mAP@5Kr, which assesses spatial alignment. The results highlight NeuroHash's ability to achieve competitive performance while offering significant advantages in flexibility and customization, paving the way for more advanced and versatile image retrieval systems.
Paper Structure (25 sections, 9 equations, 3 figures, 3 tables)

This paper contains 25 sections, 9 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Hash-based Approximate Nearest Search
  4. Deep Hashing Approach
  5. Hyperdimensional Computing
  6. Methodology
  7. HDC Basics
  8. Proposed Framework
  9. Overall Pipeline
  10. Global and Local Visual Features Extraction
  11. Context-aware HDC Encoding
  12. Hyperdimensional Spatial-aware Encoding
  13. Multilinear Hyperplane Hashing Optimization
  14. Loss term for numerical correspondence.
  15. Loss terms for limited representation.
  16. ...and 10 more sections

Figures (3)

  • Figure 1: The actual image retrieval results comparing our framework's (a) spatial-aware retrieval and (b) conditional retrieval with (c) baseline retrieval.
  • Figure 2: Overall pipeline of our proposed NeuroHash a novel neuro-symbolic framework for spatial-aware hashing and conditional image retrieval in a self-supervised manner.
  • Figure 3: Qualitative evaluation of NeuroHash on (a) Spatial-aware Retrieval and (b) Spatial-aware Conditional Image Retrieval.