Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Contextualization with SPLADE for High Recall Retrieval

Eugene Yang

TL;DR

This work tackles high recall retrieval (HRR) in Technology-Assisted Review (TAR) by marrying contextualized sparse features from SPLADE with efficient linear models. It treats HRR as a transductive, iterative learning problem and uses SPLADE to provide sparse, contextualized document representations that feed logistic regression, achieving notable reductions in review costs at a fixed recall target (80%). The study demonstrates that SPLADE alone offers moderate gains over BM25 and that combining SPLADE with BM25 yields the largest cost reductions (up to 27%) across one-phase and two-phase TAR workflows on two HRR datasets (RCV1-v2 and Jeb Bush). Key findings include the inefficiency of cross-encoders in TAR, the value of retrieval-focused SPLADE tuning, and the robust benefits of integrating contextualized sparse features into HRR pipelines, making HRR more scalable and cost-effective in practice.

Abstract

High Recall Retrieval (HRR), such as eDiscovery and medical systematic review, is a search problem that optimizes the cost of retrieving most relevant documents in a given collection. Iterative approaches, such as iterative relevance feedback and uncertainty sampling, are shown to be effective under various operational scenarios. Despite neural models demonstrating success in other text-related tasks, linear models such as logistic regression, in general, are still more effective and efficient in HRR since the model is trained and retrieves documents from the same fixed collection. In this work, we leverage SPLADE, an efficient retrieval model that transforms documents into contextualized sparse vectors, for HRR. Our approach combines the best of both worlds, leveraging both the contextualization from pretrained language models and the efficiency of linear models. It reduces 10% and 18% of the review cost in two HRR evaluation collections under a one-phase review workflow with a target recall of 80%. The experiment is implemented with TARexp and is available at https://github.com/eugene-yang/LSR-for-TAR.

Contextualization with SPLADE for High Recall Retrieval

TL;DR

This work tackles high recall retrieval (HRR) in Technology-Assisted Review (TAR) by marrying contextualized sparse features from SPLADE with efficient linear models. It treats HRR as a transductive, iterative learning problem and uses SPLADE to provide sparse, contextualized document representations that feed logistic regression, achieving notable reductions in review costs at a fixed recall target (80%). The study demonstrates that SPLADE alone offers moderate gains over BM25 and that combining SPLADE with BM25 yields the largest cost reductions (up to 27%) across one-phase and two-phase TAR workflows on two HRR datasets (RCV1-v2 and Jeb Bush). Key findings include the inefficiency of cross-encoders in TAR, the value of retrieval-focused SPLADE tuning, and the robust benefits of integrating contextualized sparse features into HRR pipelines, making HRR more scalable and cost-effective in practice.

Abstract

High Recall Retrieval (HRR), such as eDiscovery and medical systematic review, is a search problem that optimizes the cost of retrieving most relevant documents in a given collection. Iterative approaches, such as iterative relevance feedback and uncertainty sampling, are shown to be effective under various operational scenarios. Despite neural models demonstrating success in other text-related tasks, linear models such as logistic regression, in general, are still more effective and efficient in HRR since the model is trained and retrieves documents from the same fixed collection. In this work, we leverage SPLADE, an efficient retrieval model that transforms documents into contextualized sparse vectors, for HRR. Our approach combines the best of both worlds, leveraging both the contextualization from pretrained language models and the efficiency of linear models. It reduces 10% and 18% of the review cost in two HRR evaluation collections under a one-phase review workflow with a target recall of 80%. The experiment is implemented with TARexp and is available at https://github.com/eugene-yang/LSR-for-TAR.
Paper Structure (9 sections, 1 figure, 3 tables)

This paper contains 9 sections, 1 figure, 3 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Contextualized Text Features with Learned-Sparse Retrieval
  4. Experiments
  5. Results and Analysis
  6. Other Language Models
  7. Category Analysis on RCV1
  8. Cost Dynamics with SPLADE features
  9. Summary

Figures (1)

  • Figure 1: Cost dynamic graphs on category COMMERCIAL VEHICLES(I35102), which is a hard and rare category in RCV, using a two-phase workflow with uncertainty sampling under an expensive training cost structure. The gray dashed vertical line indicates where the iteration requires no second phase review, i.e., first phase review already reaches 80% recall.