Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Conspiracy Frame: a Semiotically-Driven Approach for Conspiracy Theories Detection

Heidi Campana Piva, Shaina Ashraf, Maziar Kianimoghadam Jouneghani, Arianna Longo, Rossana Damiano, Lucie Flek, Marco Antonio Stranisci

Abstract

Conspiracy theories are anti-authoritarian narratives that lead to social conflict, impacting how people perceive political information. To help in understanding this issue, we introduce the Conspiracy Frame: a fine-grained semantic representation of conspiratorial narratives derived from frame-semantics and semiotics, which spawned the Conspiracy Frames (Con.Fra.) dataset: a corpus of Telegram messages annotated at span-level. The Conspiracy Frame and Con.Fra. dataset contribute to the implementation of a more generalizable understanding and recognition of conspiracy theories. We observe the ability of LLMs to recognize this phenomenon in-domain and out-of-domain, investigating the role that frames may have in supporting this task. Results show that, while the injection of frames in an in-context approach does not lead to clear increase of performance, it has potential; the mapping of annotated spans with FrameNet shows abstract semantic patterns (e.g., `Kinship', `Ingest\_substance') that potentially pave the way for a more semantically- and semiotically-aware detection of conspiratorial narratives.

Conspiracy Frame: a Semiotically-Driven Approach for Conspiracy Theories Detection

Abstract

Conspiracy theories are anti-authoritarian narratives that lead to social conflict, impacting how people perceive political information. To help in understanding this issue, we introduce the Conspiracy Frame: a fine-grained semantic representation of conspiratorial narratives derived from frame-semantics and semiotics, which spawned the Conspiracy Frames (Con.Fra.) dataset: a corpus of Telegram messages annotated at span-level. The Conspiracy Frame and Con.Fra. dataset contribute to the implementation of a more generalizable understanding and recognition of conspiracy theories. We observe the ability of LLMs to recognize this phenomenon in-domain and out-of-domain, investigating the role that frames may have in supporting this task. Results show that, while the injection of frames in an in-context approach does not lead to clear increase of performance, it has potential; the mapping of annotated spans with FrameNet shows abstract semantic patterns (e.g., `Kinship', `Ingest\_substance') that potentially pave the way for a more semantically- and semiotically-aware detection of conspiratorial narratives.
Paper Structure (24 sections, 3 figures)

This paper contains 24 sections, 3 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. The Conspiracy Frames Corpus
  4. Data collection
  5. The Annotation Scheme: Designing the Conspiracy Frame
  6. The Annotation Workshop
  7. First session: corpus creation.
  8. Second session: LLM evaluation.
  9. Experimental Setting
  10. Aligning the Conspiracy Frame with FrameNet.
  11. Frame-driven CT recognition.
  12. Evaluation Metrics.
  13. Qualitative analysis of most recurring frames.
  14. Results
  15. Automatic CT Detection
  16. ...and 9 more sections

Figures (3)

  • Figure 1: Performance Analysis: Model Size and Prompt Strategy Effects
  • Figure 2: Span-Level F1 Score Comparison: All 70B Models
  • Figure 3: The ELO scores of the Few-Shot and Frame-Guided settings. Figure on the left shows the comparison between of the two approaches in CT recognition; Figure on the right on span detection.