Prompt Injection Vulnerability of Consensus Generating Applications in Digital Democracy

TL;DR

This work investigates prompt-injection vulnerabilities in LLM-based consensus generation for digital democracy. It introduces a four-dimensional taxonomy of attacks and evaluates vulnerability across multiple models (LLaMA 3.1 8B Instruct, GPT-4.1 Nano, Apertus 8B Instruct) using a UK deliberation dataset and Swiss Smartvote data. The study demonstrates that attacker-prompts—especially completion and criticism variants targeting ambiguous consensus—can substantially shift outputs, and proposes Direct Preference Optimization (DPO) plus layered defenses (SecAlign++, Defensive Tokens, DataFilter, Deliberative Alignment, GRPO) to enhance robustness. Findings show that DPO and defenses reduce but do not eliminate vulnerability, with model-dependent effectiveness; the Swiss dataset confirms generalization to larger, real-world contexts. Overall, the paper highlights critical security considerations for deploying consensus-generating LLMs in digital democracy and points to directions for more secure deliberation systems.

Abstract

Large Language Models (LLMs) are gaining traction as a method to generate consensus statements and aggregate preferences in digital democracy experiments. Yet, LLMs could introduce critical vulnerabilities in these systems. Here, we explore the vulnerability of some off-the-shelf LLMs to prompt-injection attacks in consensus generating systems using a four-dimensional taxonomy of attacks. In LLaMA 3.1 8B and Chat GPT 4.1 Nano, we find LLMs to be more vulnerable to attacks using disagreeable prompts and when targeting situations with unclear consensus. We also find evidence of more effective manipulation when using explicit imperatives and rational-sounding arguments compared to emotional language or fabricated statistics. To mitigate these vulnerabilities, we apply Direct Preference Optimization (DPO), an alignment method that fine-tunes LLMs to prefer unperturbed consensus statements. While DPO and additional layered defenses significantly improve robustness, it still offers limited protection against attacks targeting ambiguous consensus. These results advance our understanding of the vulnerability and robustness of consensus generating LLMs in digital democracy applications.

Figures (16)

• Figure 1: (a) Prompt-injection in a consensus statement generation example from tessler2024ai, tessler2024ai Example of a prompt injection. (b) Examples of prompt-injections following our proposed taxonomy: Manually (left) and machine readable (right) prompt-injections; Ignore (left) and Completion (right) prompt-injections; Framing, with support (left) and criticism (right) attacks; Rhetorical Strategy, composed by five manipulation strategies, each strategy grouping eight injection texts. In our experiments, we substitute the red text based on the topic being discussed.
• Figure 2: Process of introducing a prompt attack in the consensus generation and evaluating the consensus change.
• Figure 3: Process of introducing a prompt attack in the consensus generation and evaluating the consensus change.
• Figure 4: Effectiveness of prompt-injection attacks by taxonomy dimensions in LLaMA 3.1 8B Instruct: ignore/completion, framing, and rhetorical strategy. See Appendix F for results with GPT 4.1 Nano and Apertus 8B.
• Figure 5: Robustness to prompt-injection attacks via DPO by taxonomy dimensions: ignore/completion and framing.