Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DS SERVE: A Framework for Efficient and Scalable Neural Retrieval

Jinjian Liu, Yichuan Wang, Xinxi Lyu, Rulin Shao, Joseph E. Gonzalez, Matei Zaharia, Sewon Min

TL;DR

DS-Serve is presented, a framework that transforms large-scale text datasets, comprising half a trillion tokens, into a high-performance neural retrieval system that supports inference-time trade-offs between latency, accuracy, and result diversity.

Abstract

We present DS-Serve, a framework that transforms large-scale text datasets, comprising half a trillion tokens, into a high-performance neural retrieval system. DS-Serve offers both a web interface and API endpoints, achieving low latency with modest memory overhead on a single node. The framework also supports inference-time trade-offs between latency, accuracy, and result diversity. We anticipate that DS-Serve will be broadly useful for a range of applications, including large-scale retrieval-augmented generation (RAG), training data attribution, training search agents, and beyond.

DS SERVE: A Framework for Efficient and Scalable Neural Retrieval

TL;DR

DS-Serve is presented, a framework that transforms large-scale text datasets, comprising half a trillion tokens, into a high-performance neural retrieval system that supports inference-time trade-offs between latency, accuracy, and result diversity.

Abstract

We present DS-Serve, a framework that transforms large-scale text datasets, comprising half a trillion tokens, into a high-performance neural retrieval system. DS-Serve offers both a web interface and API endpoints, achieving low latency with modest memory overhead on a single node. The framework also supports inference-time trade-offs between latency, accuracy, and result diversity. We anticipate that DS-Serve will be broadly useful for a range of applications, including large-scale retrieval-augmented generation (RAG), training data attribution, training search agents, and beyond.
Paper Structure (9 sections, 1 equation, 1 figure, 1 table)

This paper contains 9 sections, 1 equation, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Description of DS Serve
  3. DS Serve Backend
  4. Datastore.
  5. Approximate Nearest Neighbor (ANN) Search.
  6. Exact Search.
  7. Diverse Search.
  8. Interface Design
  9. Evaluation and Application

Figures (1)

  • Figure 1: DS Serve converts a large dataset into a neural retrieval system: a query $q$ retrieves relevant text via ANN (DiskANN or IVFPQ), optionally reranks with Exact and/or Diverse Search, and returns the top-$k$ chunks with voting for feedback.