Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Embedding-Based Context-Aware Reranker

Ye Yuan, Mohammad Amin Shabani, Siqi Liu

TL;DR

This paper introduces Embedding-Based Context-Aware Reranker (EBCAR), a lightweight reranker that operates entirely on dense passage embeddings to enable fast, scalable inference. By augmenting passage embeddings with document IDs and positional encodings and applying a Transformer with a hybrid attention mechanism (shared full attention plus document-local masked attention), EBCAR models both inter-document and intra-document context to support cross-passage inference. Trained with a contrastive InfoNCE objective, EBCAR achieves strong ranking performance on the ConTEB benchmark, particularly on tasks demanding entity disambiguation and coreference resolution, while delivering significantly higher throughput than text-based or large-LM rerankers. The work demonstrates that embedding-space reranking with structural signals is a practical, accurate alternative for RAG pipelines in settings where cross-passage evidence must be aggregated across documents.

Abstract

Retrieval-Augmented Generation (RAG) systems rely on retrieving relevant evidence from a corpus to support downstream generation. The common practice of splitting a long document into multiple shorter passages enables finer-grained and targeted information retrieval. However, it also introduces challenges when a correct retrieval would require inference across passages, such as resolving coreference, disambiguating entities, and aggregating evidence scattered across multiple sources. Many state-of-the-art (SOTA) reranking methods, despite utilizing powerful large pretrained language models with potentially high inference costs, still neglect the aforementioned challenges. Therefore, we propose Embedding-Based Context-Aware Reranker (EBCAR), a lightweight reranking framework operating directly on embeddings of retrieved passages with enhanced cross-passage understandings through the structural information of the passages and a hybrid attention mechanism, which captures both high-level interactions across documents and low-level relationships within each document. We evaluate EBCAR against SOTA rerankers on the ConTEB benchmark, demonstrating its effectiveness for information retrieval requiring cross-passage inference and its advantages in both accuracy and efficiency.

Embedding-Based Context-Aware Reranker

TL;DR

This paper introduces Embedding-Based Context-Aware Reranker (EBCAR), a lightweight reranker that operates entirely on dense passage embeddings to enable fast, scalable inference. By augmenting passage embeddings with document IDs and positional encodings and applying a Transformer with a hybrid attention mechanism (shared full attention plus document-local masked attention), EBCAR models both inter-document and intra-document context to support cross-passage inference. Trained with a contrastive InfoNCE objective, EBCAR achieves strong ranking performance on the ConTEB benchmark, particularly on tasks demanding entity disambiguation and coreference resolution, while delivering significantly higher throughput than text-based or large-LM rerankers. The work demonstrates that embedding-space reranking with structural signals is a practical, accurate alternative for RAG pipelines in settings where cross-passage evidence must be aggregated across documents.

Abstract

Retrieval-Augmented Generation (RAG) systems rely on retrieving relevant evidence from a corpus to support downstream generation. The common practice of splitting a long document into multiple shorter passages enables finer-grained and targeted information retrieval. However, it also introduces challenges when a correct retrieval would require inference across passages, such as resolving coreference, disambiguating entities, and aggregating evidence scattered across multiple sources. Many state-of-the-art (SOTA) reranking methods, despite utilizing powerful large pretrained language models with potentially high inference costs, still neglect the aforementioned challenges. Therefore, we propose Embedding-Based Context-Aware Reranker (EBCAR), a lightweight reranking framework operating directly on embeddings of retrieved passages with enhanced cross-passage understandings through the structural information of the passages and a hybrid attention mechanism, which captures both high-level interactions across documents and low-level relationships within each document. We evaluate EBCAR against SOTA rerankers on the ConTEB benchmark, demonstrating its effectiveness for information retrieval requiring cross-passage inference and its advantages in both accuracy and efficiency.
Paper Structure (29 sections, 2 equations, 4 figures, 4 tables)

This paper contains 29 sections, 2 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Contextual Retrieval.
  4. Reranking Methods.
  5. Evaluation Settings.
  6. Methodology
  7. Embedding-Based Framework
  8. Embedding-Based Context-Aware Reranker
  9. Architecture.
  10. Training Objective.
  11. Inference.
  12. Experiments
  13. Datasets and Evaluation Protocol
  14. Training Procedure.
  15. Test Datasets.
  16. ...and 14 more sections

Figures (4)

  • Figure 1: A Retrieval-Augmented Generation (RAG) pipeline with passage-level retrieval. Long documents are chunked into passages before being embedded into a dense vector store. At query time, top-$k$ passages are retrieved and reranked to provide evidence for downstream generation.
  • Figure 2: Overview of the EBCAR architecture. Candidate passage embeddings are enriched with document ID and passage position information, then processed jointly with the query embedding through a Transformer encoder. The hybrid attention mechanism combines a shared full attention module and a document-aware masked attention module.
  • Figure 3: (a) For each method, we compute the gap in nDCG@$10$ from the best-performing method on each dataset. The boxes represent the interquartile range ($25$th to $75$th percentile), vertical lines and black triangles indicate the median and mean, respectively. (b) An overall effectiveness-and-efficiency comparison across all methods. The effectiveness is measured by the average nDCG@$10$ across $8$ test datasets. The efficiency is quantified by the throughput.
  • Figure 4: Impact of retrieved candidate size.