Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Revisiting Bi-Encoder Neural Search: An Encoding--Searching Separation Perspective

Hung-Nghiep Tran, Akiko Aizawa, Atsuhiro Takasu

TL;DR

The paper addresses limitations of the prevalent bi-encoder neural search, notably the encoding information bottleneck and the embedding-for-search assumption, which hinder performance and generalization. It develops a thought-provoking encoding--searching separation perspective that decouples the encoding and task-specific searching into generic encoding and a specialized searching module, creating an actionable encoding gap. Through theoretical analysis and a thought experiment, it identifies root causes of weaknesses and outlines mitigations such as relocating bottlenecks to the searching component and using precomputed, generic encodings to enable transfer and efficiency. The proposed framework introduces a new design surface for neural retrieval, with potential practical impact on training cost, zero-shot performance, and adaptability across tasks and modalities, while inviting empirical validation and further research.

Abstract

This paper reviews, analyzes, and proposes a new perspective on the bi-encoder architecture for neural search. While the bi-encoder architecture is widely used due to its simplicity and scalability at test time, it has some notable issues such as low performance on seen datasets and weak zero-shot performance on new datasets. In this paper, we analyze these issues and summarize two main critiques: the encoding information bottleneck problem and limitations of the basic assumption of embedding search. We then construct a thought experiment to logically analyze the encoding and searching operations and challenge the basic assumptions of embedding search. Building on these observations, we propose a new perspective on the bi-encoder architecture called the \textit{encoding--searching separation} perspective, which conceptually and practically separates the encoding and searching operations. This framework is applied to explain the root cause of existing issues and suggest mitigation strategies, potentially lowering training costs and improving retrieval performance. Finally, we discuss the broader implications of the ideas underlying this perspective, the new design surface it exposes, and potential research directions arising from it.

Revisiting Bi-Encoder Neural Search: An Encoding--Searching Separation Perspective

TL;DR

The paper addresses limitations of the prevalent bi-encoder neural search, notably the encoding information bottleneck and the embedding-for-search assumption, which hinder performance and generalization. It develops a thought-provoking encoding--searching separation perspective that decouples the encoding and task-specific searching into generic encoding and a specialized searching module, creating an actionable encoding gap. Through theoretical analysis and a thought experiment, it identifies root causes of weaknesses and outlines mitigations such as relocating bottlenecks to the searching component and using precomputed, generic encodings to enable transfer and efficiency. The proposed framework introduces a new design surface for neural retrieval, with potential practical impact on training cost, zero-shot performance, and adaptability across tasks and modalities, while inviting empirical validation and further research.

Abstract

This paper reviews, analyzes, and proposes a new perspective on the bi-encoder architecture for neural search. While the bi-encoder architecture is widely used due to its simplicity and scalability at test time, it has some notable issues such as low performance on seen datasets and weak zero-shot performance on new datasets. In this paper, we analyze these issues and summarize two main critiques: the encoding information bottleneck problem and limitations of the basic assumption of embedding search. We then construct a thought experiment to logically analyze the encoding and searching operations and challenge the basic assumptions of embedding search. Building on these observations, we propose a new perspective on the bi-encoder architecture called the \textit{encoding--searching separation} perspective, which conceptually and practically separates the encoding and searching operations. This framework is applied to explain the root cause of existing issues and suggest mitigation strategies, potentially lowering training costs and improving retrieval performance. Finally, we discuss the broader implications of the ideas underlying this perspective, the new design surface it exposes, and potential research directions arising from it.
Paper Structure (28 sections, 4 equations, 1 figure)

This paper contains 28 sections, 4 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Background
  3. Search Task
  4. Search Approaches
  5. Bi-Encoder Architecture
  6. Critiques of the Bi-Encoder Architecture
  7. Analyses and Main Critiques of Bi-Encoder Architecture
  8. The Encoding Information Bottleneck Problem
  9. Limitations of the Basic Assumption of Embedding Search
  10. New Perspective on the Bi-Encoder Architecture
  11. Thought Experiment
  12. The Encoding--Searching Separation Perspective
  13. Discussions
  14. Main Implications
  15. Revisit the Issues and Critiques of the Bi-encoder Architecture
  16. ...and 13 more sections

Figures (1)

  • Figure 1: Illustration of the Encoding--Searching Separation perspective and the induced "encoding gap", which is the difference of the information between the encoding and searching tasks. Note the change from specific encoding to generic encoding.