Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Negative Sampling Techniques in Information Retrieval: A Survey

Laurin Wischounig, Abdelrahman Abdallah, Adam Jatowt

Abstract

Information Retrieval (IR) is fundamental to many modern NLP applications. The rise of dense retrieval (DR), using neural networks to learn semantic vector representations, has significantly advanced IR performance. Central to training effective dense retrievers through contrastive learning is the selection of informative negative samples. Synthesizing 35 seminal papers, this survey provides a comprehensive and up-to-date overview of negative sampling techniques in dense IR. Our unique contribution is the focus on modern NLP applications and the inclusion of recent Large Language Model (LLM)-driven methods, an area absent in prior reviews. We propose a taxonomy that categorizes techniques including random, static/dynamically mined, and synthetic datasets. We then analyze these approaches with respect to trade-offs between effectiveness, computational cost, and implementation difficulty. The survey concludes by outlining current challenges and promising future directions for the use of LLM-generated synthetic data.

Negative Sampling Techniques in Information Retrieval: A Survey

Abstract

Information Retrieval (IR) is fundamental to many modern NLP applications. The rise of dense retrieval (DR), using neural networks to learn semantic vector representations, has significantly advanced IR performance. Central to training effective dense retrievers through contrastive learning is the selection of informative negative samples. Synthesizing 35 seminal papers, this survey provides a comprehensive and up-to-date overview of negative sampling techniques in dense IR. Our unique contribution is the focus on modern NLP applications and the inclusion of recent Large Language Model (LLM)-driven methods, an area absent in prior reviews. We propose a taxonomy that categorizes techniques including random, static/dynamically mined, and synthetic datasets. We then analyze these approaches with respect to trade-offs between effectiveness, computational cost, and implementation difficulty. The survey concludes by outlining current challenges and promising future directions for the use of LLM-generated synthetic data.
Paper Structure (59 sections, 1 equation, 1 figure, 11 tables)

This paper contains 59 sections, 1 equation, 1 figure, 11 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Contrastive Learning for Dense Representations
  4. Taxonomy of Negative Sampling Techniques
  5. Sampling Techniques
  6. Random and In-Batch Negatives
  7. Static Hard Negative Mining
  8. Dynamic Hard Negative Mining
  9. Cluster-Based Mining
  10. Principled Sampling with TriSampler
  11. False Negative Mitigation
  12. Filtering and Denoising Negatives
  13. Robustness through Regularization
  14. Data-Centric Methods
  15. Data Augmentation
  16. ...and 44 more sections

Figures (1)

  • Figure 1: Taxonomy of negative sampling techniques for dense retrieval. The framework divides approaches into two main categories: Sampling-based techniques (orange) and Data-centric techniques (cyan)