Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Stochastic Indexing Primitives for Non-Deterministic Molecular Archives

Faruk Alpay, Levent Sarioglu

TL;DR

This work addresses the challenge of scalable, content-addressable random access in DNA-based data archives. It introduces the Holographic Bloom Filter (HBF), a high-dimensional, associative index that binds key and value vectors via circular convolution and stores their superposed bindings in a single memory vector, enabling one-shot retrieval by correlation with a query key. The authors provide explicit construction and decoding algorithms, a signal–noise decomposition, and concentration/extreme-value analyses that yield threshold and margin settings with exponential error decay in the vector dimension under explicit noise models. They compare HBF to pointer-chasing baselines, discuss design trade-offs, robustness, and potential integration with traditional error correction, highlighting substantial reductions in retrieval latency and energy by leveraging parallel molecular interactions. Overall, HBF offers a concrete, analyzable framework for content-addressable indexing in DNA storage, with quantified trade-offs between dimensionality, dataset size, and noise, paving the way toward practical in-situ indexing primitives for molecular archives.

Abstract

Random access remains a central bottleneck in DNA-based data storage. Existing systems typically retrieve records by PCR enrichment or other multi-step biochemical procedures, which do not naturally support fast, massively parallel, content-addressable queries. We introduce the Holographic Bloom Filter (HBF), a probabilistic indexing primitive that stores key-pointer associations as a single high-dimensional memory vector. HBF binds a key vector and a value (pointer) vector using circular convolution and superposes bindings across all records. A query decodes by correlating the memory with the query key and selecting the best matching value using a margin-based decision rule. We give construction and decoding algorithms and a probabilistic analysis under explicit noise models (memory corruption and query/key mismatches). The analysis provides concentration bounds for match and non-match score distributions, explicit threshold and margin settings for a top K decoder, and exponential error decay in the vector dimension under standard randomness assumptions. HBF offers a concrete, analyzable alternative to pointer-chasing molecular data structures, enabling one-shot associative retrieval while quantifying trade-offs among dimensionality, dataset size, and noise.

Stochastic Indexing Primitives for Non-Deterministic Molecular Archives

TL;DR

This work addresses the challenge of scalable, content-addressable random access in DNA-based data archives. It introduces the Holographic Bloom Filter (HBF), a high-dimensional, associative index that binds key and value vectors via circular convolution and stores their superposed bindings in a single memory vector, enabling one-shot retrieval by correlation with a query key. The authors provide explicit construction and decoding algorithms, a signal–noise decomposition, and concentration/extreme-value analyses that yield threshold and margin settings with exponential error decay in the vector dimension under explicit noise models. They compare HBF to pointer-chasing baselines, discuss design trade-offs, robustness, and potential integration with traditional error correction, highlighting substantial reductions in retrieval latency and energy by leveraging parallel molecular interactions. Overall, HBF offers a concrete, analyzable framework for content-addressable indexing in DNA storage, with quantified trade-offs between dimensionality, dataset size, and noise, paving the way toward practical in-situ indexing primitives for molecular archives.

Abstract

Random access remains a central bottleneck in DNA-based data storage. Existing systems typically retrieve records by PCR enrichment or other multi-step biochemical procedures, which do not naturally support fast, massively parallel, content-addressable queries. We introduce the Holographic Bloom Filter (HBF), a probabilistic indexing primitive that stores key-pointer associations as a single high-dimensional memory vector. HBF binds a key vector and a value (pointer) vector using circular convolution and superposes bindings across all records. A query decodes by correlating the memory with the query key and selecting the best matching value using a margin-based decision rule. We give construction and decoding algorithms and a probabilistic analysis under explicit noise models (memory corruption and query/key mismatches). The analysis provides concentration bounds for match and non-match score distributions, explicit threshold and margin settings for a top K decoder, and exponential error decay in the vector dimension under standard randomness assumptions. HBF offers a concrete, analyzable alternative to pointer-chasing molecular data structures, enabling one-shot associative retrieval while quantifying trade-offs among dimensionality, dataset size, and noise.
Paper Structure (32 sections, 7 theorems, 30 equations, 2 figures, 3 algorithms)

This paper contains 32 sections, 7 theorems, 30 equations, 2 figures, 3 algorithms.

Table of Contents

  1. Keywords.
  2. Introduction
  3. Problem statement.
  4. Contributions.
  5. Model and Related Work
  6. Formal Model (Indexing as Noisy Associative Decoding)
  7. DNA Data Storage and Random Access (Context)
  8. Probabilistic Data Structures
  9. Vector Symbolic Architectures (VSA)
  10. Notation and Mathematical Preliminaries
  11. HBF Design and Algorithms
  12. Data Structure Definition
  13. Insertion Algorithm
  14. Batch Construction (Practical Encoder View)
  15. Query Algorithm
  16. ...and 17 more sections

Key Result

Lemma 1

Let $u,v\in\{\pm1\}^d$ be independent. Then $\langle u,v\rangle=\sum_{j=1}^d u[j]v[j]$ satisfies

Figures (2)

  • Figure 1: High-level schematic of HBF: key/value vectors are bound by circular convolution and superposed into $M$; queries correlate $M$ with a key vector to recover the associated value.
  • Figure 2: Comparison of retrieval processes.Top: In a skip-graph, a query for key $x$ traverses intermediate nodes via sequential pointer-resolution steps. Bottom: In the HBF, a query interacts with the full memory in parallel and yields an output $y$ after a single round of interaction.

Theorems & Definitions (17)

  • Definition 1: Circular convolution and correlation
  • Remark 1: Fast implementation
  • Definition 2: Random sign model
  • Lemma 1: Inner-product concentration
  • proof
  • Proposition 1: Moments for match vs. non-match scores
  • proof
  • Theorem 1: Margin decoder succeeds at high SNR
  • proof
  • Lemma 2: Score tail bound via sub-Gaussianity
  • ...and 7 more