Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Mapping and Influencing the Political Ideology of Large Language Models using Synthetic Personas

Pietro Bernardelle, Leon Fröhling, Stefano Civelli, Riccardo Lunardi, Kevin Roitero, Gianluca Demartini

TL;DR

The paper addresses how synthetic personas influence the political orientations of large language models when evaluated with the Political Compass Test (PCT). It combines PersonaHub with a two-phase prompting strategy across four open-source LLMs to map persona-driven political distributions and to test manipulation with explicit ideological prompts. Baseline results show a left-libertarian bias in persona responses, with distinct differences across models; explicit prompts shift responses toward right-authoritarian positions, with Llama showing the strongest responsiveness and Zephyr the most resistance. The work highlights significant prompt-based malleability in LLM political expressions and underscores implications for bias mitigation, safety, and the design of debiasing strategies in real-world deployments.

Abstract

The analysis of political biases in large language models (LLMs) has primarily examined these systems as single entities with fixed viewpoints. While various methods exist for measuring such biases, the impact of persona-based prompting on LLMs' political orientation remains unexplored. In this work we leverage PersonaHub, a collection of synthetic persona descriptions, to map the political distribution of persona-based prompted LLMs using the Political Compass Test (PCT). We then examine whether these initial compass distributions can be manipulated through explicit ideological prompting towards diametrically opposed political orientations: right-authoritarian and left-libertarian. Our experiments reveal that synthetic personas predominantly cluster in the left-libertarian quadrant, with models demonstrating varying degrees of responsiveness when prompted with explicit ideological descriptors. While all models demonstrate significant shifts towards right-authoritarian positions, they exhibit more limited shifts towards left-libertarian positions, suggesting an asymmetric response to ideological manipulation that may reflect inherent biases in model training.

Mapping and Influencing the Political Ideology of Large Language Models using Synthetic Personas

TL;DR

The paper addresses how synthetic personas influence the political orientations of large language models when evaluated with the Political Compass Test (PCT). It combines PersonaHub with a two-phase prompting strategy across four open-source LLMs to map persona-driven political distributions and to test manipulation with explicit ideological prompts. Baseline results show a left-libertarian bias in persona responses, with distinct differences across models; explicit prompts shift responses toward right-authoritarian positions, with Llama showing the strongest responsiveness and Zephyr the most resistance. The work highlights significant prompt-based malleability in LLM political expressions and underscores implications for bias mitigation, safety, and the design of debiasing strategies in real-world deployments.

Abstract

The analysis of political biases in large language models (LLMs) has primarily examined these systems as single entities with fixed viewpoints. While various methods exist for measuring such biases, the impact of persona-based prompting on LLMs' political orientation remains unexplored. In this work we leverage PersonaHub, a collection of synthetic persona descriptions, to map the political distribution of persona-based prompted LLMs using the Political Compass Test (PCT). We then examine whether these initial compass distributions can be manipulated through explicit ideological prompting towards diametrically opposed political orientations: right-authoritarian and left-libertarian. Our experiments reveal that synthetic personas predominantly cluster in the left-libertarian quadrant, with models demonstrating varying degrees of responsiveness when prompted with explicit ideological descriptors. While all models demonstrate significant shifts towards right-authoritarian positions, they exhibit more limited shifts towards left-libertarian positions, suggesting an asymmetric response to ideological manipulation that may reflect inherent biases in model training.

Paper Structure

This paper contains 16 sections, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. RELATED WORK
  3. Personas in LLMs
  4. LLMs & Political Compass Test
  5. METHODOLOGY
  6. Data
  7. Models
  8. Experimental setup
  9. Computational resources
  10. RESULTS
  11. Personas political distribution
  12. Personas manipulation
  13. CONCLUSION
  14. Limitations
  15. Future studies
  16. ...and 1 more sections

Figures (2)

  • Figure 1: Political compass distribution of PersonaHub personas when impersonated by different LLMs. Darker regions indicate higher density of personas on a logarithmic scale. The white dot represents the leaning of the original LLM (without any form of persona prompting). The bar charts show the marginal distributions along each axis.
  • Figure 2: Political compass distribution of PersonaHub personas when impersonated by different LLMs. Top: Distribution after injecting the "right-authoritarian" descriptor. Bottom: Distribution after injecting the "left-libertarian" descriptor. Darker regions indicate higher density of personas on a logarithmic scale. The white triangle shows the average political position of the model across all persona-based prompts without explicit descriptor injection (i.e., the centroid from Figure \ref{['fig:combined']}).