Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

NeedleDB: A Generative-AI Based System for Accurate and Efficient Image Retrieval using Complex Natural Language Queries

Mahdi Erfanian, Abolfazl Asudeh

Abstract

We demonstrate NeedleDB, an open-source, deployment-ready database system for answering complex natural language queries over image data. Unlike existing approaches that rely on contrastive-learning embeddings (e.g., CLIP), which degrade on compositional or nuanced queries, NeedleDB leverages generative AI to synthesize guide images that represent the query in the visual domain, transforming the text-to-image retrieval problem into a more tractable image-to-image search. The system aggregates nearest-neighbor results across multiple vision embedders using a weighted rank-fusion strategy grounded in a Monte Carlo estimator with provable error bounds. NeedleDB ships with a full-featured command-line interface (needlectl), a browser-based Web UI, and a modular microservice architecture backed by PostgreSQL and Milvus. On challenging benchmarks, it improves Mean Average Precision by up to 93% over the strongest baseline while maintaining sub-second query latency. In our demonstration, attendees interact with NeedleDB through three hands-on scenarios that showcase its retrieval capabilities, data ingestion workflow, and pipeline configurability.

NeedleDB: A Generative-AI Based System for Accurate and Efficient Image Retrieval using Complex Natural Language Queries

Abstract

We demonstrate NeedleDB, an open-source, deployment-ready database system for answering complex natural language queries over image data. Unlike existing approaches that rely on contrastive-learning embeddings (e.g., CLIP), which degrade on compositional or nuanced queries, NeedleDB leverages generative AI to synthesize guide images that represent the query in the visual domain, transforming the text-to-image retrieval problem into a more tractable image-to-image search. The system aggregates nearest-neighbor results across multiple vision embedders using a weighted rank-fusion strategy grounded in a Monte Carlo estimator with provable error bounds. NeedleDB ships with a full-featured command-line interface (needlectl), a browser-based Web UI, and a modular microservice architecture backed by PostgreSQL and Milvus. On challenging benchmarks, it improves Mean Average Precision by up to 93% over the strongest baseline while maintaining sub-second query latency. In our demonstration, attendees interact with NeedleDB through three hands-on scenarios that showcase its retrieval capabilities, data ingestion workflow, and pipeline configurability.

Paper Structure

This paper contains 11 sections, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. System Architecture
  3. Preprocessing (Indexing)
  4. Inference Pipeline
  5. System Features
  6. Demonstration Scenarios
  7. Scenario 1: Simple vs. Complex Queries
  8. Scenario 2: Index Your Own Dataset
  9. Scenario 3: Under the Hood
  10. Experimental Highlights
  11. Conclusion

Figures (4)

  • Figure 1: Top-3 results on BDD100K yu2020bdd100k for queries of increasing complexity. CLIP returns relevant results only for the simple query (a), whereas NeedleDB retrieves relevant images for all three queries using just 2 guide images (RealVisXL) and 2 embedders (EVA, RegNet)
  • Figure 2: Architecture of the NeedleDB system. A query flows from the user layer through the backend's multi-stage inference pipeline to the infrastructure services.
  • Figure 3: The NeedleDB Web UI: (a) search page showing generated guide images alongside retrieved results, (b) system status dashboard with API health and generator availability, (c) directory management with live indexing progress, and (d) generator configuration with drag-to-reorder priority.
  • Figure 4: The needlectl command-line interface: (a) query execution with per-embedder verbose results, (b) service status showing Docker containers and Python microservices, (c) Textual-based TUI for directory management, and (d) TUI for generator priority and activation.