LEANN: A Low-Storage Vector Index

TL;DR

The paper addresses the prohibitive storage overhead of embedding-based vector indices by introducing LEANN, a storage-efficient index that recomputes embeddings on demand and prunes index metadata. It combines a two-level search with hybrid distance using selective exact recomputation and dynamic batching to maintain accuracy while reducing latency. A high-degree preserving graph pruning strategy and a sharded merging index-building pipeline further shrink storage and enable efficient updates. Empirical results show LEANN delivers up to 50x storage reduction with recall comparable to state-of-the-art methods and acceptable end-to-end latency for RAG on personal devices, enabling practical on-device vector search for privacy-preserving and large-scale retrieval tasks.

Abstract

Embedding-based vector search underpins many important applications, such as recommendation and retrieval-augmented generation (RAG). It relies on vector indices to enable efficient search. However, these indices require storing high-dimensional embeddings and large index metadata, whose total size can be several times larger than the original data (e.g., text chunks). Such high storage overhead makes it difficult, or even impractical, to deploy vector search on personal devices or large-scale datasets. To tackle this problem, we propose LEANN, a storage-efficient index for vector search that recomputes embeddings on the fly instead of storing them, and compresses state-of-the-art proximity graph indices while preserving search accuracy. LEANN delivers high-quality vector search while using only a fraction of the storage (e.g., 5% of the original data) and supporting storage-efficient index construction and updates. On real-world benchmarks, LEANN reduces index size by up to 50x compared with conventional indices, while maintaining SOTA accuracy and comparable latency for RAG applications.

Figures (13)

• Figure 1: LEANN System Diagram. The system combines high-degree preserving graph pruning for minimal storage footprint with graph-based recomputation and two-level search with dynamic batching for efficient query processing (Steps 1-4).
• Figure 2: HNSW graph analysis reveals skewed access and degree distributions, with node degrees capped at 60 by HNSW.
• Figure 3: Storage consumption of different vector index methods on the RPJ-Wiki dataset. The black dashed line marks the raw dataset size (76 GB), while the red dashed line shows the typical RAM capacity (32 GB, RTX 4090, following our testbed configuration in \ref{['sec:setup']}). Memory-heavy methods like HNSW exceed this RAM limit and cannot run on such hardware. LEANN achieves the lowest storage footprint at only 5% of the original dataset size.
• Figure 4: Comparison of Exact Match and F1 scores for downstream RAG tasks across four methods: keyword search (BM25), PQ-compressed vector search, HNSW, and LEANN. HNSW and LEANN are configured to achieve a target recall of 90%, while the PQ baseline is given extended search time to reach its highest possible recall. Here we use Qwen3-4B as the generation model.
• Figure 5: Speedup achieved by different optimization techniques described in \ref{['sec:latency_optimize']} when evaluated on four datasets to reach the same recall level. Two-level refers to the optimization in \ref{['sec:doubleq']}, while Batch corresponds to \ref{['sec:batch']}.