DYNAMICQA: Tracing Internal Knowledge Conflicts in Language Models

TL;DR

It is verified that LMs exhibit a greater degree of intra-memory conflict with dynamic facts compared to facts that have a single truth value, suggesting that retrieval-augmented generation will struggle with the most commonly adapted facts.

Abstract

Knowledge-intensive language understanding tasks require Language Models (LMs) to integrate relevant context, mitigating their inherent weaknesses, such as incomplete or outdated knowledge. However, conflicting knowledge can be present in the LM's parameters, termed intra-memory conflict, which can affect a model's propensity to accept contextual knowledge. To study the effect of intra-memory conflict on an LM's ability to accept relevant context, we utilize two knowledge conflict measures and a novel dataset containing inherently conflicting data, DynamicQA. This dataset includes facts with a temporal dynamic nature where facts can change over time and disputable dynamic facts, which can change depending on the viewpoint. DynamicQA is the first to include real-world knowledge conflicts and provide context to study the link between the different types of knowledge conflicts. We also evaluate several measures on their ability to reflect the presence of intra-memory conflict: semantic entropy and a novel coherent persuasion score. With our extensive experiments, we verify that LMs exhibit a greater degree of intra-memory conflict with dynamic facts compared to facts that have a single truth value. Furthermore, we reveal that facts with intra-memory conflict are harder to update with context, suggesting that retrieval-augmented generation will struggle with the most commonly adapted facts.

Figures (6)

• Figure 1: We present examples of our dataset here, consisting of static, temporal, and disputable facts. We show how model output distribution can vary due to the popularity (a,b) and dynamicity (c) of facts. Fact dynamicity (c) causes intra-memory conflicts between the different fact representations seen during pretraining. In the bottom row, we show the change in output probability (purple area) that the context must enact on the initial output distribution (dotted line) to force a new output distribution (purple line).
• Figure 2: The distribution of loss ($\mathcal{L}(a_c,x_{i;q})$) for generating a particular answer ($a_c$) given only the question ($x_{i;q}$) for each partition of the dataset.
• Figure 3: The instance-level relationship between Coherent Persuasion score (§ \ref{['sec:2:context']}) and 3 possible factors that impact persuasion: semantic entropy, subject popularity and object temporality, alongside their Pearson correlation scores. Temporality shows the strongest relationship with persuasion. We also highlight two behaviours of interest: persuaded and stubborn instances.
• Figure 4: The distribution of losses for each partition of the dataset on Mistral and Qwen
• Figure 5: The instance-level relationship between Coherent Persuasion Score and 3 possible factors that impact persuasion: semantic entropy, subject popularity and object temporality, alongside their Pearson correlation scores. Temporality shows the strongest relationship with persuasion. We also highlight two behaviours of interest: persuaded and stubborn instances. These results are for Mistral.