MicroNN: An On-device Disk-resident Updatable Vector Database
Jeffrey Pound, Floris Chabert, Arjun Bhushan, Ankur Goswami, Anil Pacaci, Shihabur Rahman Chowdhury
TL;DR
MicroNN tackles the challenge of on-device vector similarity search under tight memory constraints while supporting continuous updates and hybrid queries that couple vector search with structured attributes. It introduces a disk-resident IVF index built via mini-batch k-means, a delta-store for streaming updates, and a relational storage layer (SQLite) to ensure durability and ACID semantics, plus a hybrid query optimizer and multi-query processing. Empirical results show MicroNN achieving near in-memory latency for top-100 ANN with 90% recall on million-scale data while using around 10 MB of memory, and demonstrate scalable index construction with significantly lower memory and strong update support. The work enables robust on-device search for privacy-preserving, offline, and batch-intensive workloads, reducing I/O and enabling practical production use-cases on resource-constrained devices.
Abstract
Nearest neighbour search over dense vector collections has important applications in information retrieval, retrieval augmented generation (RAG), and content ranking. Performing efficient search over large vector collections is a well studied problem with many existing approaches and open source implementations. However, most state-of-the-art systems are generally targeted towards scenarios using large servers with an abundance of memory, static vector collections that are not updatable, and nearest neighbour search in isolation of other search criteria. We present Micro Nearest Neighbour (MicroNN), an embedded nearest-neighbour vector search engine designed for scalable similarity search in low-resource environments. MicroNN addresses the problem of on-device vector search for real-world workloads containing updates and hybrid search queries that combine nearest neighbour search with structured attribute filters. In this scenario, memory is highly constrained and disk-efficient index structures and algorithms are required, as well as support for continuous inserts and deletes. MicroNN is an embeddable library that can scale to large vector collections with minimal resources. MicroNN is used in production and powers a wide range of vector search use-cases on-device. MicroNN takes less than 7 ms to retrieve the top-100 nearest neighbours with 90% recall on publicly available million-scale vector benchmark while using ~10 MB of memory.