Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Embark on DenseQuest: A System for Selecting the Best Dense Retriever for a Custom Collection

Ekaterina Khramtsova, Teerapong Leelanupab, Shengyao Zhuang, Mahsa Baktashmotlagh, Guido Zuccon

TL;DR

DenseQuest provides a scalable, cloud-based solution to select the optimal dense retriever for private, unlabeled collections using unsupervised performance evaluation. By consolidating multiple per-query and per-collection DR selection methods into a two-container web app, it enables end-to-end deployment from collection upload to model ranking and checkpoint download. The system encodes collections across a pool of dense retrievers, then ranks them with diverse unsupervised metrics such as Binary Entropy, Query Alteration, Score-based QPP, Fusion-based QPP, MSMARCO/MTEB rankings, and LARMOR, with GPU-accelerated cores for certain methods. Its architecture supports scalable infrastructure, asynchronous task processing, and flexible deployment, offering a practical tool for IR engineers to tailor dense retrieval to private datasets without obtaining relevance judgments. The work highlights the value of integrating state-of-the-art DR selection techniques into an accessible platform, potentially accelerating private-domain IR deployment and benchmarking.

Abstract

In this demo we present a web-based application for selecting an effective pre-trained dense retriever to use on a private collection. Our system, DenseQuest, provides unsupervised selection and ranking capabilities to predict the best dense retriever among a pool of available dense retrievers, tailored to an uploaded target collection. DenseQuest implements a number of existing approaches, including a recent, highly effective method powered by Large Language Models (LLMs), which requires neither queries nor relevance judgments. The system is designed to be intuitive and easy to use for those information retrieval engineers and researchers who need to identify a general-purpose dense retrieval model to encode or search a new private target collection. Our demonstration illustrates conceptual architecture and the different use case scenarios of the system implemented on the cloud, enabling universal access and use. DenseQuest is available at https://densequest.ielab.io.

Embark on DenseQuest: A System for Selecting the Best Dense Retriever for a Custom Collection

TL;DR

DenseQuest provides a scalable, cloud-based solution to select the optimal dense retriever for private, unlabeled collections using unsupervised performance evaluation. By consolidating multiple per-query and per-collection DR selection methods into a two-container web app, it enables end-to-end deployment from collection upload to model ranking and checkpoint download. The system encodes collections across a pool of dense retrievers, then ranks them with diverse unsupervised metrics such as Binary Entropy, Query Alteration, Score-based QPP, Fusion-based QPP, MSMARCO/MTEB rankings, and LARMOR, with GPU-accelerated cores for certain methods. Its architecture supports scalable infrastructure, asynchronous task processing, and flexible deployment, offering a practical tool for IR engineers to tailor dense retrieval to private datasets without obtaining relevance judgments. The work highlights the value of integrating state-of-the-art DR selection techniques into an accessible platform, potentially accelerating private-domain IR deployment and benchmarking.

Abstract

In this demo we present a web-based application for selecting an effective pre-trained dense retriever to use on a private collection. Our system, DenseQuest, provides unsupervised selection and ranking capabilities to predict the best dense retriever among a pool of available dense retrievers, tailored to an uploaded target collection. DenseQuest implements a number of existing approaches, including a recent, highly effective method powered by Large Language Models (LLMs), which requires neither queries nor relevance judgments. The system is designed to be intuitive and easy to use for those information retrieval engineers and researchers who need to identify a general-purpose dense retrieval model to encode or search a new private target collection. Our demonstration illustrates conceptual architecture and the different use case scenarios of the system implemented on the cloud, enabling universal access and use. DenseQuest is available at https://densequest.ielab.io.
Paper Structure (10 sections, 3 figures, 1 table)

This paper contains 10 sections, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. DenseQuest
  3. Architecture and Workflow
  4. Infrastructure
  5. Task Queue Management
  6. The Core of DenseQuest
  7. Demonstration
  8. Utility and Usage/Implication
  9. Future Functions
  10. Conclusion

Figures (3)

  • Figure 1: Overview of the architectural components of DenseQuest
  • Figure 2: DenseQuest result page for a target collection uploaded by a user. Note that for publication purposes, the layout of the UI presented here has been modified and differs slightly from that of an actual UI, while retaining all functionalities.
  • Figure 3: DenseQuest job/result list, as uploaded by a user.