Tevatron: An Efficient and Flexible Toolkit for Dense Retrieval

TL;DR

Tevatron addresses the need for a flexible, efficient dense retrieval software stack by providing a modular toolkit that unifies data handling, model training, encoding, and retrieval. It supports PyTorch and JAX, integrates FAISS for indexing, and emphasizes memory-efficient data management and scalable training through features like gradient caching. Empirical results show Tevatron can match or exceed DPR performance while reducing memory usage and enabling faster training, with demonstrated success in cross-lingual and supervised IR settings. The CLI-driven workflow and self-contained datasets lower barriers for rapid experimentation, making Tevatron a practical foundation for ongoing dense retrieval research.

Abstract

Recent rapid advancements in deep pre-trained language models and the introductions of large datasets have powered research in embedding-based dense retrieval. While several good research papers have emerged, many of them come with their own software stacks. These stacks are typically optimized for some particular research goals instead of efficiency or code structure. In this paper, we present Tevatron, a dense retrieval toolkit optimized for efficiency, flexibility, and code simplicity. Tevatron provides a standardized pipeline for dense retrieval including text processing, model training, corpus/query encoding, and search. This paper presents an overview of Tevatron and demonstrates its effectiveness and efficiency across several IR and QA data sets. We also show how Tevatron's flexible design enables easy generalization across datasets, model architectures, and accelerator platforms(GPU/TPU). We believe Tevatron can serve as an effective software foundation for dense retrieval system research including design, modeling, and optimization.

Figures (2)

• Figure 1: Tevatron raw data template for IR datasets.
• Figure 2: Training process of Tevatron (Pytorch)