Robust Information Retrieval

TL;DR

The paper addresses the robustness of information retrieval systems beyond average effectiveness, focusing on adversarial robustness and out-of-distribution generalization, while acknowledging new challenges and opportunities introduced by large language models. It presents a structured tutorial that surveys taxonomy, attack/defense methods, and OOD strategies, and discusses robustness in the age of LLMs. Key contributions include a consolidated synthesis of IR robustness progress, a taxonomy of adversarial and OOD issues, and guidance on benchmarks, datasets, and promising future directions. The work aims to guide researchers and practitioners toward more reliable IR in diverse real-world settings and to inspire unified evaluation efforts and robust IR benchmarks.

Abstract

Beyond effectiveness, the robustness of an information retrieval (IR) system is increasingly attracting attention. When deployed, a critical technology such as IR should not only deliver strong performance on average but also have the ability to handle a variety of exceptional situations. In recent years, research into the robustness of IR has seen significant growth, with numerous researchers offering extensive analyses and proposing myriad strategies to address robustness challenges. In this tutorial, we first provide background information covering the basics and a taxonomy of robustness in IR. Then, we examine adversarial robustness and out-of-distribution (OOD) robustness within IR-specific contexts, extensively reviewing recent progress in methods to enhance robustness. The tutorial concludes with a discussion on the robustness of IR in the context of large language models (LLMs), highlighting ongoing challenges and promising directions for future research. This tutorial aims to generate broader attention to robustness issues in IR, facilitate an understanding of the relevant literature, and lower the barrier to entry for interested researchers and practitioners.