Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Lying with Truths: Open-Channel Multi-Agent Collusion for Belief Manipulation via Generative Montage

Jinwei Hu, Xinmiao Huang, Youcheng Sun, Yi Dong, Xiaowei Huang

TL;DR

This work identifies a novel cognitive collusion threat in which autonomous LLM-based agents can be steered to adopt fabricated beliefs using only truthful evidence fragments distributed publicly. It formalizes the Cognitive Collusion Attack and introduces Generative Montage, a Writer-Editor-Director framework that constructs adversarial narratives from truthful fragments and coordinates distribution via Sybil publishers. Using CoPHEME, a dataset derived from real rumor events, the authors show pervasive vulnerability across 14 LLM families, with attack success rates above 70% in many cases and higher susceptibility among reasoning-enabled models. The study also demonstrates that manipulated beliefs cascade to downstream decision-makers, yielding high downstream deception rates despite verification mechanisms, highlighting a socio-technical vulnerability in agent-driven information ecosystems and underscoring the need for provenance-aware defenses and robust evaluation frameworks.

Abstract

As large language models (LLMs) transition to autonomous agents synthesizing real-time information, their reasoning capabilities introduce an unexpected attack surface. This paper introduces a novel threat where colluding agents steer victim beliefs using only truthful evidence fragments distributed through public channels, without relying on covert communications, backdoors, or falsified documents. By exploiting LLMs' overthinking tendency, we formalize the first cognitive collusion attack and propose Generative Montage: a Writer-Editor-Director framework that constructs deceptive narratives through adversarial debate and coordinated posting of evidence fragments, causing victims to internalize and propagate fabricated conclusions. To study this risk, we develop CoPHEME, a dataset derived from real-world rumor events, and simulate attacks across diverse LLM families. Our results show pervasive vulnerability across 14 LLM families: attack success rates reach 74.4% for proprietary models and 70.6% for open-weights models. Counterintuitively, stronger reasoning capabilities increase susceptibility, with reasoning-specialized models showing higher attack success than base models or prompts. Furthermore, these false beliefs then cascade to downstream judges, achieving over 60% deception rates, highlighting a socio-technical vulnerability in how LLM-based agents interact with dynamic information environments. Our implementation and data are available at: https://github.com/CharlesJW222/Lying_with_Truth/tree/main.

Lying with Truths: Open-Channel Multi-Agent Collusion for Belief Manipulation via Generative Montage

TL;DR

This work identifies a novel cognitive collusion threat in which autonomous LLM-based agents can be steered to adopt fabricated beliefs using only truthful evidence fragments distributed publicly. It formalizes the Cognitive Collusion Attack and introduces Generative Montage, a Writer-Editor-Director framework that constructs adversarial narratives from truthful fragments and coordinates distribution via Sybil publishers. Using CoPHEME, a dataset derived from real rumor events, the authors show pervasive vulnerability across 14 LLM families, with attack success rates above 70% in many cases and higher susceptibility among reasoning-enabled models. The study also demonstrates that manipulated beliefs cascade to downstream decision-makers, yielding high downstream deception rates despite verification mechanisms, highlighting a socio-technical vulnerability in agent-driven information ecosystems and underscoring the need for provenance-aware defenses and robust evaluation frameworks.

Abstract

As large language models (LLMs) transition to autonomous agents synthesizing real-time information, their reasoning capabilities introduce an unexpected attack surface. This paper introduces a novel threat where colluding agents steer victim beliefs using only truthful evidence fragments distributed through public channels, without relying on covert communications, backdoors, or falsified documents. By exploiting LLMs' overthinking tendency, we formalize the first cognitive collusion attack and propose Generative Montage: a Writer-Editor-Director framework that constructs deceptive narratives through adversarial debate and coordinated posting of evidence fragments, causing victims to internalize and propagate fabricated conclusions. To study this risk, we develop CoPHEME, a dataset derived from real-world rumor events, and simulate attacks across diverse LLM families. Our results show pervasive vulnerability across 14 LLM families: attack success rates reach 74.4% for proprietary models and 70.6% for open-weights models. Counterintuitively, stronger reasoning capabilities increase susceptibility, with reasoning-specialized models showing higher attack success than base models or prompts. Furthermore, these false beliefs then cascade to downstream judges, achieving over 60% deception rates, highlighting a socio-technical vulnerability in how LLM-based agents interact with dynamic information environments. Our implementation and data are available at: https://github.com/CharlesJW222/Lying_with_Truth/tree/main.
Paper Structure (48 sections, 12 equations, 4 figures, 6 tables, 2 algorithms)

This paper contains 48 sections, 12 equations, 4 figures, 6 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. The Illusion of Causality in LLMs
  4. Collusion Threat in Multi-Agent Systems
  5. Problem Formulation
  6. Preliminaries
  7. Probabilistic Vulnerability Modeling
  8. The Cognitive Collusion Problem
  9. Methodology
  10. Explicit Collusion
  11. Adversarial Narrative Production
  12. Writer ($\mathcal{A}_W$): Narrative Synthesis.
  13. Editor ($\mathcal{A}_E$): Montage Sequencing.
  14. Director ($\mathcal{A}_D$): Adversarial Debate.
  15. Decentralized Injection via Publisher
  16. ...and 33 more sections

Figures (4)

  • Figure 1: Collusion via hidden channels (left) versus belief steering via public, truthful evidence (right).
  • Figure 2: Generative Montage Framework. (1) Production Team constructs deceptive narratives from truthful fragments via adversarial debate; (2) Sybil Publishers distribute curated fragments publicly; (3) Victim Agents independently internalize fabricated beliefs; (4) Downstream Judges aggregate contaminated analysis from multiple benign victims and ratify them as facts. Explicit colluders (1-2) intentionally deceive; implicit colluders (3-4) unwittingly amplify misinformation.
  • Figure 3: Downstream Deception Rate Analysis. Event-level DDR heatmap under Majority Vote (left) and AI Judge (middle) strategies, with aggregated comparison across model families (right).
  • Figure 4: Effectiveness Across Sequence Lengths.

Theorems & Definitions (3)

  • Definition 1: Local Truth Constraint
  • Definition 2: Global Lie Condition
  • Definition 3: Colluder