Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

How Easily do Irrelevant Inputs Skew the Responses of Large Language Models?

Siye Wu, Jian Xie, Jiangjie Chen, Tinghui Zhu, Kai Zhang, Yanghua Xiao

TL;DR

This work tackles how irrelevant inputs can skew LLM responses in retrieval-augmented systems by constructing a graded set of irrelevant information with semantic ties to the questions. It introduces a methodology to generate Unrelated, Partially Related, and Related content using Wikipedia passages and a dense retriever, and evaluates LLMs on curated QA datasets (PopQA, EntityQuestions) across multiple models. The study reveals that highly semantically related irrelevant information can substantially mislead LLMs, that increasing the amount of irrelevant content worsens discrimination, and that current mitigation strategies offer only marginal or even detrimental benefits. The findings highlight the need for robust RAG pipelines and targeted strategies to filter semantically related distractions, with resources available on GitHub.

Abstract

By leveraging the retrieval of information from external knowledge databases, Large Language Models (LLMs) exhibit enhanced capabilities for accomplishing many knowledge-intensive tasks. However, due to the inherent flaws of current retrieval systems, there might exist irrelevant information within those retrieving top-ranked passages. In this work, we present a comprehensive investigation into the robustness of LLMs to different types of irrelevant information under various conditions. We initially introduce a framework to construct high-quality irrelevant information that ranges from semantically unrelated, partially related, and related to questions. Furthermore, our analysis demonstrates that the constructed irrelevant information not only scores highly on similarity metrics, being highly retrieved by existing systems, but also bears semantic connections to the context. Our investigation reveals that current LLMs still face challenges in discriminating highly semantically related information and can be easily distracted by these irrelevant yet misleading content. Besides, we also find that current solutions for handling irrelevant information have limitations in improving the robustness of LLMs to such distractions. All the resources are available on GitHub at https://github.com/Di-viner/LLM-Robustness-to-Irrelevant-Information.

How Easily do Irrelevant Inputs Skew the Responses of Large Language Models?

TL;DR

This work tackles how irrelevant inputs can skew LLM responses in retrieval-augmented systems by constructing a graded set of irrelevant information with semantic ties to the questions. It introduces a methodology to generate Unrelated, Partially Related, and Related content using Wikipedia passages and a dense retriever, and evaluates LLMs on curated QA datasets (PopQA, EntityQuestions) across multiple models. The study reveals that highly semantically related irrelevant information can substantially mislead LLMs, that increasing the amount of irrelevant content worsens discrimination, and that current mitigation strategies offer only marginal or even detrimental benefits. The findings highlight the need for robust RAG pipelines and targeted strategies to filter semantically related distractions, with resources available on GitHub.

Abstract

By leveraging the retrieval of information from external knowledge databases, Large Language Models (LLMs) exhibit enhanced capabilities for accomplishing many knowledge-intensive tasks. However, due to the inherent flaws of current retrieval systems, there might exist irrelevant information within those retrieving top-ranked passages. In this work, we present a comprehensive investigation into the robustness of LLMs to different types of irrelevant information under various conditions. We initially introduce a framework to construct high-quality irrelevant information that ranges from semantically unrelated, partially related, and related to questions. Furthermore, our analysis demonstrates that the constructed irrelevant information not only scores highly on similarity metrics, being highly retrieved by existing systems, but also bears semantic connections to the context. Our investigation reveals that current LLMs still face challenges in discriminating highly semantically related information and can be easily distracted by these irrelevant yet misleading content. Besides, we also find that current solutions for handling irrelevant information have limitations in improving the robustness of LLMs to such distractions. All the resources are available on GitHub at https://github.com/Di-viner/LLM-Robustness-to-Irrelevant-Information.
Paper Structure (32 sections, 7 figures, 16 tables)

This paper contains 32 sections, 7 figures, 16 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Retrieval-Augmented Generation
  4. Robustness to Irrelevant Information
  5. Experimental Setup
  6. Datasets
  7. Parametric Memory Elicitation
  8. Information Categorization
  9. Irrelevant Information Construction
  10. Unrelated Information
  11. Partially Related Information
  12. Related Information
  13. Information Quality Measurement
  14. Evaluation Metrics
  15. Experiments and Analysis
  16. ...and 17 more sections

Figures (7)

  • Figure 1: An example of how semantically related irrelevant information distracts LLMs. LLMs are misled by the information due to over-reasoning.
  • Figure 2: A tree of information categorization in this paper. "Gold Passage" indicates the passage containing gold answers.
  • Figure 3: Distribution of similarity scores across different information types for PopQA, with "Wiki Pass." indicating the Top 1 Wikipedia passage specifically retrieved by the Contriever model.
  • Figure 4: Top-scored variant proportions in Related Information for PopQA. "MLink", "CChar", and "FAnec" indicate "Misleading Linkage", "Common Characteristics", and "Fictional Anecdotes", respectively.
  • Figure 5: Solution attempts in a more complicated scenario, with all information presented to GPT-3.5 Turbo. Specifically, "+CoT" indicates prompting the model to think step by step, "+instr." adds an instruction "feel free to ignore irrelevant information" into the prompt, and "+ICL" refers to adding examples to guide LLMs in discerning irrelevant information. For a discussion about the model-based fine-tuning method, please refer to Appendix \ref{['subsec: result of ft method']}.
  • ...and 2 more figures