Studying Large Language Model Behaviors Under Context-Memory Conflicts With Real Documents

TL;DR

It is found that knowledge updates fail less often than previously reported, and the factual parametric knowledge of LLMs can negatively influence their reading abilities and behaviors.

Abstract

Retrieval-augmented generation (RAG) mitigates many problems of fully parametric language models, such as temporal degradation, hallucinations, and lack of grounding. In RAG, the model's knowledge can be updated from documents provided in context. This leads to cases of conflict between the model's parametric knowledge and the contextual information, where the model may not always update its knowledge. Previous work studied context-memory knowledge conflicts by creating synthetic documents that contradict the model's correct parametric answers. We present a framework for studying such knowledge conflicts in a realistic setup. We update incorrect parametric knowledge using real conflicting documents. This reflects how knowledge conflicts arise in practice. In this realistic scenario, we find that knowledge updates fail less often than previously reported. In cases where the models still fail to update their answers, we find a parametric bias: the incorrect parametric answer appearing in context makes the knowledge update likelier to fail. These results suggest that the factual parametric knowledge of LLMs can negatively influence their reading abilities and behaviors. Our code is available at https://github.com/kortukov/realistic_knowledge_conflicts/ .

Figures (3)

• Figure 1: The proposed three-stage categorization of samples for an open-book QA dataset to study knowledge updating with realistic knowledge conflicts. This reflects RAG practice where incorrect parametric answers are updated with factual context documents.
• Figure 2: Proportion of examples with incorrect parametric answer in context ($a_p \subseteq c$) in the full knowledge conflict (KC) dataset, the Retain subset ($\mathbf{R}$) and the Correct update subset ($\mathbf{U_c}$) (as defined in \ref{['subsec:formal_notation']}) for Llama2-7B.
• Figure 3: Frequency of samples that contain the incorrect parametric answer in the context ($a_p \subseteq c$) in the full knowledge conflict (KC) data, the Retain subset ($\mathbf{R}$) and the Correct update subset ($\mathbf{U_c}$)for each studied LLM. Across all datasets and models, the percentage of context documents containing the incorrect parametric answer is largest in $\mathbf{R}$.

Theorems & Definitions (2)

• Definition 1
• Definition 2