Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

From HNSW to Information-Theoretic Binarization: Rethinking the Architecture of Scalable Vector Search

Seyed Moein Abtahi, Majid Fekri, Tara Khani, Akramul Azim

TL;DR

The work addresses the high cost of in-memory vector search driven by the HNSW+float32+cosine stack and the ineffectiveness of common cost-cutting tactics. It proposes an information-theoretic serverless paradigm built on Maximally Informative Binarization, Efficient Distance Metric, and Information-Theoretic Score, enabling exhaustive search over compact binary representations with deterministic results. Empirical evidence on MAIR shows competitive retrieval quality relative to full-precision baselines while delivering lower latency and scalable throughput, supported by a total cost of ownership analysis favoring per-query pricing. This approach signals a potential paradigm shift away from the in-memory index monopoly toward scalable, cloud-native vector search with strong semantic performance guarantees.

Abstract

Modern semantic search and retrieval-augmented generation (RAG) systems rely predominantly on in-memory approximate nearest neighbor (ANN) indexes over high-precision floating-point vectors, resulting in escalating operational cost and inherent trade-offs between latency, throughput, and retrieval accuracy. This paper analyzes the architectural limitations of the dominant "HNSW + float32 + cosine similarity" stack and evaluates existing cost-reduction strategies, including storage disaggregation and lossy vector quantization, which inevitably sacrifice either performance or accuracy. We introduce and empirically evaluate an alternative information-theoretic architecture based on maximally informative binarization (MIB), efficient bitwise distance metrics, and an information-theoretic scoring (ITS) mechanism. Unlike conventional ANN systems, this approach enables exhaustive search over compact binary representations, allowing deterministic retrieval and eliminating accuracy degradation under high query concurrency. Using the MAIR benchmark across 14 datasets and 10,038 queries, we compare this architecture against Elasticsearch, Pinecone, PGVector, and Qdrant. Results demonstrate retrieval quality comparable to full-precision systems, while achieving substantially lower latency and maintaining constant throughput at high request rates. We show that this architectural shift enables a truly serverless, cost-per-query deployment model, challenging the necessity of large in-memory ANN indexes for high-quality semantic search.

From HNSW to Information-Theoretic Binarization: Rethinking the Architecture of Scalable Vector Search

TL;DR

The work addresses the high cost of in-memory vector search driven by the HNSW+float32+cosine stack and the ineffectiveness of common cost-cutting tactics. It proposes an information-theoretic serverless paradigm built on Maximally Informative Binarization, Efficient Distance Metric, and Information-Theoretic Score, enabling exhaustive search over compact binary representations with deterministic results. Empirical evidence on MAIR shows competitive retrieval quality relative to full-precision baselines while delivering lower latency and scalable throughput, supported by a total cost of ownership analysis favoring per-query pricing. This approach signals a potential paradigm shift away from the in-memory index monopoly toward scalable, cloud-native vector search with strong semantic performance guarantees.

Abstract

Modern semantic search and retrieval-augmented generation (RAG) systems rely predominantly on in-memory approximate nearest neighbor (ANN) indexes over high-precision floating-point vectors, resulting in escalating operational cost and inherent trade-offs between latency, throughput, and retrieval accuracy. This paper analyzes the architectural limitations of the dominant "HNSW + float32 + cosine similarity" stack and evaluates existing cost-reduction strategies, including storage disaggregation and lossy vector quantization, which inevitably sacrifice either performance or accuracy. We introduce and empirically evaluate an alternative information-theoretic architecture based on maximally informative binarization (MIB), efficient bitwise distance metrics, and an information-theoretic scoring (ITS) mechanism. Unlike conventional ANN systems, this approach enables exhaustive search over compact binary representations, allowing deterministic retrieval and eliminating accuracy degradation under high query concurrency. Using the MAIR benchmark across 14 datasets and 10,038 queries, we compare this architecture against Elasticsearch, Pinecone, PGVector, and Qdrant. Results demonstrate retrieval quality comparable to full-precision systems, while achieving substantially lower latency and maintaining constant throughput at high request rates. We show that this architectural shift enables a truly serverless, cost-per-query deployment model, challenging the necessity of large in-memory ANN indexes for high-quality semantic search.
Paper Structure (81 sections, 4 equations, 5 figures, 3 tables)

This paper contains 81 sections, 4 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. The Vector Search Cost-Performance Problem
  3. Architectural Foundation and Cost Implications
  4. The Three-Component Algorithmic Stack
  5. Index Structure: HNSW
  6. Numerical Representation: Float32 Vectors
  7. Distance Metric: Cosine Similarity
  8. Cost Structure Analysis
  9. Existing Cost Mitigation Strategies: Compromise Architectures
  10. Storage Disaggregation: The Latency-Cost Trade-off
  11. Vector Quantization: The Accuracy-Cost Trade-off
  12. Information-Theoretic Vector Search: A Novel Paradigm
  13. Theoretical Foundation and Motivation
  14. Architecture: Information-Theoretic Stack
  15. Maximally Informative Binarization (MIB)
  16. ...and 66 more sections

Figures (5)

  • Figure 1: Radar chart comparing quantized vector search performance across 10 datasets: AILA2019-Statutes, AILA2019-Case (Legal & Regulatory), FiQA, FinanceBench (Financial), LegalQuAD, REGIR-EU2UK, REGIR-UK2EU (Legal & Regulatory), HC3Finance (Financial), NFCorpus (Medical & Clinical), and LeCaRDv2 (Legal & Regulatory).
  • Figure 2: Radar chart comparing vector search performance across 8 datasets: AILA2019-Statutes, FiQA (Financial), REGIR-EU2UK (Legal & Regulatory), ConvFinQA, HC3Finance (Financial), LeCaRDv2 (Legal & Regulatory), AILA2019-Case (Legal & Regulatory), and Apple (API Documentation).
  • Figure 3: Latency time comparison in milliseconds (ms) across 14 datasets ordered by dataset size for four different systems: Moorcheh Distance Only, Elasticsearch All, PGVector All, and Qdrant All. Qdrant, deployed on a resource-constrained cloud instance (0.5 vCPU, 1 GB RAM), shows higher latency due to network overhead and limited computational resources, averaging 86.79 ms across all datasets compared to Moorcheh's 9.6 ms, Elasticsearch's 10.2 ms, and PGVector's 37.3 ms.
  • Figure 4: Latency comparison across 14 MAIR datasets. Black bars: Moorcheh (all server-side components). Stacked bars: Pinecone (Red: vector search, Orange: Cohere re-rank).
  • Figure 5: Query throughput versus retrieval precision trade-off across six vector search platforms.