Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Propaganda is all you need

Paul Kronlund-Drouault

TL;DR

Propaganda is all you need investigates how alignment processes shape the political biases and embedding spaces of large language models. It combines unsupervised, ORPO/DPO, and guarded alignment perspectives with a multimodal evaluation framework that yields relative and absolute measures of ideological leaning, aiming to reveal how dominant ideology may be encoded in AI systems. The work provides a Marxian and Kuhnian theoretical lens to interpret alignment dynamics, discusses emergent misalignment, and highlights socio-political risks including intensified private control over political discourse and the need for regulation. Overall, it argues for interdisciplinary study at the intersection of AI, politics, and sociology to understand and mitigate AI-driven ideological influence on society.

Abstract

As Machine Learning (ML) is still a recent field of study, especially outside the realm of abstract Mathematics and Computer Science, few works have been conducted on the political aspect of large Language Models (LLMs), and more particularly about the alignment process and its political dimension. This process can be as simple as prompt engineering but is also very complex and can affect completely unrelated notions. For example, politically directed alignment has a very strong impact on an LLM's embedding space and the relative position of political notions in such a space. Using special tools to evaluate general political bias and analyze the effects of alignment, we can gather new data to understand its causes and possible consequences on society. Indeed, by taking a socio-political approach, we can hypothesize that most big LLMs are aligned with what Marxist philosophy calls the 'dominant ideology.' As AI's role in political decision-making, at the citizen's scale but also in government agencies, such biases can have huge effects on societal change, either by creating new and insidious pathways for societal uniformity or by allowing disguised extremist views to gain traction among the people.

Propaganda is all you need

TL;DR

Propaganda is all you need investigates how alignment processes shape the political biases and embedding spaces of large language models. It combines unsupervised, ORPO/DPO, and guarded alignment perspectives with a multimodal evaluation framework that yields relative and absolute measures of ideological leaning, aiming to reveal how dominant ideology may be encoded in AI systems. The work provides a Marxian and Kuhnian theoretical lens to interpret alignment dynamics, discusses emergent misalignment, and highlights socio-political risks including intensified private control over political discourse and the need for regulation. Overall, it argues for interdisciplinary study at the intersection of AI, politics, and sociology to understand and mitigate AI-driven ideological influence on society.

Abstract

As Machine Learning (ML) is still a recent field of study, especially outside the realm of abstract Mathematics and Computer Science, few works have been conducted on the political aspect of large Language Models (LLMs), and more particularly about the alignment process and its political dimension. This process can be as simple as prompt engineering but is also very complex and can affect completely unrelated notions. For example, politically directed alignment has a very strong impact on an LLM's embedding space and the relative position of political notions in such a space. Using special tools to evaluate general political bias and analyze the effects of alignment, we can gather new data to understand its causes and possible consequences on society. Indeed, by taking a socio-political approach, we can hypothesize that most big LLMs are aligned with what Marxist philosophy calls the 'dominant ideology.' As AI's role in political decision-making, at the citizen's scale but also in government agencies, such biases can have huge effects on societal change, either by creating new and insidious pathways for societal uniformity or by allowing disguised extremist views to gain traction among the people.
Paper Structure (19 sections, 6 figures)

This paper contains 19 sections, 6 figures.

Table of Contents

  1. Introduction
  2. Biases
  3. Political Alignment
  4. Unsupervised Alignment
  5. ORPO / DPO Alignment
  6. Guarded Alignment
  7. Aligned Humans
  8. Evaluation
  9. Methods
  10. Relative Position
  11. Absolute Position
  12. Socio-Political Impacts
  13. A Marxian Analysis
  14. Kuhn and AI alignement
  15. Emergent Misalignement
  16. ...and 4 more sections

Figures (6)

  • Figure 1: A GPT-4 openai2023gpt4 agent evaluating a GPT-4 respondent model.
  • Figure 2: Results on an absolute position dataset for a Llama-based model, trained using ORPO on a Marxist dataset.
  • Figure 3: $10$ most changed words after both unsupervised and ORPO alignment on a Marxist (Trotskyst) dataset. The value used here to measure change is the the distance between base/trained words in the model's embedding space.
  • Figure 4: Norm of the difference vectors of cherry-picked words in a GPT-2 model's embedding space before and after unsupervized alignment on a Marxist (Trokyst) dataset. Lower distance should mean closer meaning.
  • Figure 5: A multimodal agent (GPT-4 openai2023gpt4, Mistral mistral_llm_2023, and Claude claude_llm_2023) evaluating the Grok 2 model from LMSYS.
  • ...and 1 more figures