Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the Reproducibility of Learned Sparse Retrieval Adaptations for Long Documents

Emmanouil Georgios Lionis, Jia-Huei Ju

TL;DR

This work reproduces and analyzes the adaptation of Learned Sparse Retrieval (LSR) for long documents, focusing on segment-based aggregation and proximity-based SDM variants (ExactSDM, SoftSDM). It validates key claims from the original study, showing that the first document segment heavily dominates relevance and that ExactSDM and SoftSDM provide proximity-based gains over baseline representations, with ExactSDM often yielding the strongest improvements. Across short and long documents, Score-Max remains robust, while SDM variants sustain performance as segmentation grows and hyperparameters are tuned, though short documents favor ExactSDM and long documents benefit from SoftSDM expansion. The study provides a publicly available codebase, deepens understanding of segment dependencies, and highlights the need for careful hyperparameter optimization to maximize long-document retrieval effectiveness with LSR.

Abstract

Document retrieval is one of the most challenging tasks in Information Retrieval. It requires handling longer contexts, often resulting in higher query latency and increased computational overhead. Recently, Learned Sparse Retrieval (LSR) has emerged as a promising approach to address these challenges. Some have proposed adapting the LSR approach to longer documents by aggregating segmented document using different post-hoc methods, including n-grams and proximity scores, adjusting representations, and learning to ensemble all signals. In this study, we aim to reproduce and examine the mechanisms of adapting LSR for long documents. Our reproducibility experiments confirmed the importance of specific segments, with the first segment consistently dominating document retrieval performance. Furthermore, We re-evaluate recently proposed methods -- ExactSDM and SoftSDM -- across varying document lengths, from short (up to 2 segments) to longer (3+ segments). We also designed multiple analyses to probe the reproduced methods and shed light on the impact of global information on adapting LSR to longer contexts. The complete code and implementation for this project is available at: https://github.com/lionisakis/Reproducibilitiy-lsr-long.

On the Reproducibility of Learned Sparse Retrieval Adaptations for Long Documents

TL;DR

This work reproduces and analyzes the adaptation of Learned Sparse Retrieval (LSR) for long documents, focusing on segment-based aggregation and proximity-based SDM variants (ExactSDM, SoftSDM). It validates key claims from the original study, showing that the first document segment heavily dominates relevance and that ExactSDM and SoftSDM provide proximity-based gains over baseline representations, with ExactSDM often yielding the strongest improvements. Across short and long documents, Score-Max remains robust, while SDM variants sustain performance as segmentation grows and hyperparameters are tuned, though short documents favor ExactSDM and long documents benefit from SoftSDM expansion. The study provides a publicly available codebase, deepens understanding of segment dependencies, and highlights the need for careful hyperparameter optimization to maximize long-document retrieval effectiveness with LSR.

Abstract

Document retrieval is one of the most challenging tasks in Information Retrieval. It requires handling longer contexts, often resulting in higher query latency and increased computational overhead. Recently, Learned Sparse Retrieval (LSR) has emerged as a promising approach to address these challenges. Some have proposed adapting the LSR approach to longer documents by aggregating segmented document using different post-hoc methods, including n-grams and proximity scores, adjusting representations, and learning to ensemble all signals. In this study, we aim to reproduce and examine the mechanisms of adapting LSR for long documents. Our reproducibility experiments confirmed the importance of specific segments, with the first segment consistently dominating document retrieval performance. Furthermore, We re-evaluate recently proposed methods -- ExactSDM and SoftSDM -- across varying document lengths, from short (up to 2 segments) to longer (3+ segments). We also designed multiple analyses to probe the reproduced methods and shed light on the impact of global information on adapting LSR to longer contexts. The complete code and implementation for this project is available at: https://github.com/lionisakis/Reproducibilitiy-lsr-long.

Paper Structure

This paper contains 22 sections, 2 equations, 6 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Preliminary
  3. LSR for Document Retrieval
  4. Sequential Dependence Model
  5. Reproducibility Experiments
  6. Reproduced Methods
  7. SDM Methods.
  8. Other Baselines.
  9. Experimental Setups
  10. Datasets.
  11. Training.
  12. Retrieval Evaluation.
  13. Implementation Detail
  14. Results and Analysis
  15. Reproduced Results
  16. ...and 7 more sections

Figures (6)

  • Figure 1: The highest scoring segments for relevant and irrelevant document-query pairs
  • Figure 2: The highest scoring segments for only relevant document-query pairs.
  • Figure 4: Unique terms of 1st segment across other segments.
  • Figure 5: Intersection terms of 1st segment with other segments.
  • Figure 6: Global terms of 1st segment across multiple segments.
  • ...and 1 more figures