Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Conflicts in Texts: Data, Implications and Challenges

Siyi Liu, Dan Roth

TL;DR

This survey unifies conflicting information in NLP across data and deployment scenarios, spanning factual conflicts, opinion conflicts, annotation disagreements, and knowledge-related model interactions. It analyzes origins, implications, and mitigation strategies for conflicts arising from web texts, human annotations, and model use, highlighting approaches such as disambiguation, retrieval augmentation, and verification to improve reliability. The work leverages datasets and frameworks like AmbigQA, WhoQA, PERSPECTRUM, MultiOpEd, and SummaC to map the space of conflicts and proposes open challenges for building conflict-aware, robust, and fair NLP systems. Its contribution lies in providing a cohesive taxonomy and practical directions for reasoning over and reconciling conflicting information in real-world NLP applications.

Abstract

As NLP models become increasingly integrated into real-world applications, it becomes clear that there is a need to address the fact that models often rely on and generate conflicting information. Conflicts could reflect the complexity of situations, changes that need to be explained and dealt with, difficulties in data annotation, and mistakes in generated outputs. In all cases, disregarding the conflicts in data could result in undesired behaviors of models and undermine NLP models' reliability and trustworthiness. This survey categorizes these conflicts into three key areas: (1) natural texts on the web, where factual inconsistencies, subjective biases, and multiple perspectives introduce contradictions; (2) human-annotated data, where annotator disagreements, mistakes, and societal biases impact model training; and (3) model interactions, where hallucinations and knowledge conflicts emerge during deployment. While prior work has addressed some of these conflicts in isolation, we unify them under the broader concept of conflicting information, analyze their implications, and discuss mitigation strategies. We highlight key challenges and future directions for developing conflict-aware NLP systems that can reason over and reconcile conflicting information more effectively.

Conflicts in Texts: Data, Implications and Challenges

TL;DR

This survey unifies conflicting information in NLP across data and deployment scenarios, spanning factual conflicts, opinion conflicts, annotation disagreements, and knowledge-related model interactions. It analyzes origins, implications, and mitigation strategies for conflicts arising from web texts, human annotations, and model use, highlighting approaches such as disambiguation, retrieval augmentation, and verification to improve reliability. The work leverages datasets and frameworks like AmbigQA, WhoQA, PERSPECTRUM, MultiOpEd, and SummaC to map the space of conflicts and proposes open challenges for building conflict-aware, robust, and fair NLP systems. Its contribution lies in providing a cohesive taxonomy and practical directions for reasoning over and reconciling conflicting information in real-world NLP applications.

Abstract

As NLP models become increasingly integrated into real-world applications, it becomes clear that there is a need to address the fact that models often rely on and generate conflicting information. Conflicts could reflect the complexity of situations, changes that need to be explained and dealt with, difficulties in data annotation, and mistakes in generated outputs. In all cases, disregarding the conflicts in data could result in undesired behaviors of models and undermine NLP models' reliability and trustworthiness. This survey categorizes these conflicts into three key areas: (1) natural texts on the web, where factual inconsistencies, subjective biases, and multiple perspectives introduce contradictions; (2) human-annotated data, where annotator disagreements, mistakes, and societal biases impact model training; and (3) model interactions, where hallucinations and knowledge conflicts emerge during deployment. While prior work has addressed some of these conflicts in isolation, we unify them under the broader concept of conflicting information, analyze their implications, and discuss mitigation strategies. We highlight key challenges and future directions for developing conflict-aware NLP systems that can reason over and reconcile conflicting information more effectively.
Paper Structure (31 sections, 2 figures)

This paper contains 31 sections, 2 figures.

Table of Contents

  1. Introduction
  2. Conflicts in Natural Texts on the Web
  3. Factual Conflicts
  4. Origins
  5. Ambiguity
  6. Contradictory Evidence
  7. Implications and Mitigation
  8. Conflicts in Opinions
  9. Origins
  10. Perspectives
  11. Framing Bias
  12. Implications and Mitigation
  13. Conflicts in Human-Annotated Texts
  14. Origins
  15. Annotation Disagreement
  16. ...and 16 more sections

Figures (2)

  • Figure 1: Examples of the three different ares of conflicts discussed in this work. The first example describes a case where two different entities of the same name are found naturally on the web, the second example elaborates the annotation disagreement in a sentiment analysis task, and the third showcases a knowledge conflict between the context and memory of LLMs during model interactions.
  • Figure 2: Taxonomy of conflicts in texts.