Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Two-Step SPLADE: Simple, Efficient and Effective Approximation of SPLADE

Carlos Lassance, Hervé Dejean, Stéphane Clinchant, Nicola Tonellotto

TL;DR

This paper tackles the efficiency gap between learned sparse retrieval (LSR) models like SPLADE and traditional sparse methods by introducing a two-step retrieval strategy. It first computes an efficient approximate pass using pruned and reweighted SPLADE vectors, then rescales the top-$k$ documents with the original SPLADE representations. The approach maintains strong effectiveness on in-domain data while delivering large latency/throughput gains (up to 12x–40x improvements) and robustness in zero-shot settings across 30 datasets; it also offers a flexible trade-off at retrieval time. By staying compatible with existing inverted-index infrastructure and standard engines, the method presents a practical path toward production-ready LSR deployments where efficiency and effectiveness can be balanced at the user or system level.

Abstract

Learned sparse models such as SPLADE have successfully shown how to incorporate the benefits of state-of-the-art neural information retrieval models into the classical inverted index data structure. Despite their improvements in effectiveness, learned sparse models are not as efficient as classical sparse model such as BM25. The problem has been investigated and addressed by recently developed strategies, such as guided traversal query processing and static pruning, with different degrees of success on in-domain and out-of-domain datasets. In this work, we propose a new query processing strategy for SPLADE based on a two-step cascade. The first step uses a pruned and reweighted version of the SPLADE sparse vectors, and the second step uses the original SPLADE vectors to re-score a sample of documents retrieved in the first stage. Our extensive experiments, performed on 30 different in-domain and out-of-domain datasets, show that our proposed strategy is able to improve mean and tail response times over the original single-stage SPLADE processing by up to $30\times$ and $40\times$, respectively, for in-domain datasets, and by 12x to 25x, for mean response on out-of-domain datasets, while not incurring in statistical significant difference in 60\% of datasets.

Two-Step SPLADE: Simple, Efficient and Effective Approximation of SPLADE

TL;DR

This paper tackles the efficiency gap between learned sparse retrieval (LSR) models like SPLADE and traditional sparse methods by introducing a two-step retrieval strategy. It first computes an efficient approximate pass using pruned and reweighted SPLADE vectors, then rescales the top- documents with the original SPLADE representations. The approach maintains strong effectiveness on in-domain data while delivering large latency/throughput gains (up to 12x–40x improvements) and robustness in zero-shot settings across 30 datasets; it also offers a flexible trade-off at retrieval time. By staying compatible with existing inverted-index infrastructure and standard engines, the method presents a practical path toward production-ready LSR deployments where efficiency and effectiveness can be balanced at the user or system level.

Abstract

Learned sparse models such as SPLADE have successfully shown how to incorporate the benefits of state-of-the-art neural information retrieval models into the classical inverted index data structure. Despite their improvements in effectiveness, learned sparse models are not as efficient as classical sparse model such as BM25. The problem has been investigated and addressed by recently developed strategies, such as guided traversal query processing and static pruning, with different degrees of success on in-domain and out-of-domain datasets. In this work, we propose a new query processing strategy for SPLADE based on a two-step cascade. The first step uses a pruned and reweighted version of the SPLADE sparse vectors, and the second step uses the original SPLADE vectors to re-score a sample of documents retrieved in the first stage. Our extensive experiments, performed on 30 different in-domain and out-of-domain datasets, show that our proposed strategy is able to improve mean and tail response times over the original single-stage SPLADE processing by up to and , respectively, for in-domain datasets, and by 12x to 25x, for mean response on out-of-domain datasets, while not incurring in statistical significant difference in 60\% of datasets.
Paper Structure (19 sections, 1 equation, 3 figures, 2 tables, 2 algorithms)

This paper contains 19 sections, 1 equation, 3 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. Adapted Models (a):
  4. Adapted Search Mechanism(b):
  5. Two-Step SPLADE
  6. Indexing
  7. Retrieval
  8. Approximate step:
  9. Rescoring Step.
  10. Experiments
  11. Search Engine and Dynamic Pruning
  12. Datasets
  13. Models
  14. Approximation Validity
  15. Approximate Step Parameters
  16. ...and 4 more sections

Figures (3)

  • Figure 2: Two-Step SPLADE
  • Figure 3: Intersection of the original retrieval top-10 and the pruned retrieval. We represent the different combinations by: F: No pruning; V: Varying pruning; L: Lexically-pruned. D: Document, Q: Query. So F-Q, V-D means we vary document length and keep the query untouched.
  • Figure 4: Intersection of the original retrieval top-10 and the first approximate step. P-SPLADE means the L-Q, L-D version from the previous figure with the applied $k_1$ saturation. AvG L is the average latency for BEIR datasets that have more than 1M documents.