Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Toward Robust RALMs: Revealing the Impact of Imperfect Retrieval on Retrieval-Augmented Language Models

Seong-Il Park, Jay-Yoon Lee

TL;DR

It is revealed that RALMs often fail to identify the unanswerability or contradiction of a document set, which frequently leads to hallucinations, and it is shown that the addition of an adversary significantly degrades RALM’s performance.

Abstract

Retrieval Augmented Language Models (RALMs) have gained significant attention for their ability to generate accurate answer and improve efficiency. However, RALMs are inherently vulnerable to imperfect information due to their reliance on the imperfect retriever or knowledge source. We identify three common scenarios-unanswerable, adversarial, conflicting-where retrieved document sets can confuse RALM with plausible real-world examples. We present the first comprehensive investigation to assess how well RALMs detect and handle such problematic scenarios. Among these scenarios, to systematically examine adversarial robustness we propose a new adversarial attack method, Generative model-based ADVersarial attack (GenADV) and a novel metric Robustness under Additional Document (RAD). Our findings reveal that RALMs often fail to identify the unanswerability or contradiction of a document set, which frequently leads to hallucinations. Moreover, we show the addition of an adversary significantly degrades RALM's performance, with the model becoming even more vulnerable when the two scenarios overlap (adversarial+unanswerable). Our research identifies critical areas for assessing and enhancing the robustness of RALMs, laying the foundation for the development of more robust models.

Toward Robust RALMs: Revealing the Impact of Imperfect Retrieval on Retrieval-Augmented Language Models

TL;DR

It is revealed that RALMs often fail to identify the unanswerability or contradiction of a document set, which frequently leads to hallucinations, and it is shown that the addition of an adversary significantly degrades RALM’s performance.

Abstract

Retrieval Augmented Language Models (RALMs) have gained significant attention for their ability to generate accurate answer and improve efficiency. However, RALMs are inherently vulnerable to imperfect information due to their reliance on the imperfect retriever or knowledge source. We identify three common scenarios-unanswerable, adversarial, conflicting-where retrieved document sets can confuse RALM with plausible real-world examples. We present the first comprehensive investigation to assess how well RALMs detect and handle such problematic scenarios. Among these scenarios, to systematically examine adversarial robustness we propose a new adversarial attack method, Generative model-based ADVersarial attack (GenADV) and a novel metric Robustness under Additional Document (RAD). Our findings reveal that RALMs often fail to identify the unanswerability or contradiction of a document set, which frequently leads to hallucinations. Moreover, we show the addition of an adversary significantly degrades RALM's performance, with the model becoming even more vulnerable when the two scenarios overlap (adversarial+unanswerable). Our research identifies critical areas for assessing and enhancing the robustness of RALMs, laying the foundation for the development of more robust models.

Paper Structure

This paper contains 22 sections, 2 equations, 3 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Problem Setup
  4. Definition of RALM
  5. Types of imperfect documents
  6. Experimental Setup
  7. Task and Datasets
  8. Metrics
  9. Models
  10. Prompting
  11. Experiments
  12. Baseline QA Performance as a reference
  13. Identifying Unanswerability
  14. Results and Analysis
  15. Robustness on Adversarial Information
  16. ...and 7 more sections

Figures (3)

  • Figure 1: Examples of scenarios with imperfect information. A Robust RALM system can be resilient to imperfections inherent in search engines or knowledge sources.
  • Figure 2: Experimental results on identifying unanswerable examples. The x axis represents the models (size). Acc means accuracy for all examples, Acc (ans) means accuracy for answerable examples and Acc (unans) means accuracy for unanswerable examples. The two models on the far right represent the results of experiments conducted on the largest models within their respective families.
  • Figure 3: Experimental results on the effects of adding documents. The y-axis represents RAD score. GenADV refers to adding an adversarial document, Random to adding a randomly selected document, and Top-k to adding the sixth highest-ranked document.