On the steerability of large language models toward data-driven personas

TL;DR

This work addresses bias in LLM outputs by introducing data-driven personas learned from collaborative filtering, enabling steerable generation toward diverse viewpoints. A soft-prompting model maps persona embeddings to persona-specific virtual tokens that steer the LLM when prepended to inputs, allowing both individual and cluster-level control. On OpinionQA, data-driven personas significantly improve alignment with individuals' opinions (57–77% over baselines) and outperform demographic-based steering, demonstrating nuanced representations of social groups beyond demographics. The framework offers a scalable, ethical approach to balancing perspectives in LLMs, with potential to reduce polarization while highlighting limitations and future research directions.

Abstract

Large language models (LLMs) are known to generate biased responses where the opinions of certain groups and populations are underrepresented. Here, we present a novel approach to achieve controllable generation of specific viewpoints using LLMs, that can be leveraged to produce multiple perspectives and to reflect the diverse opinions. Moving beyond the traditional reliance on demographics like age, gender, or party affiliation, we introduce a data-driven notion of persona grounded in collaborative filtering, which is defined as either a single individual or a cohort of individuals manifesting similar views across specific inquiries. As individuals in the same demographic group may have different personas, our data-driven persona definition allows for a more nuanced understanding of different (latent) social groups present in the population. In addition to this, we also explore an efficient method to steer LLMs toward the personas that we define. We show that our data-driven personas significantly enhance model steerability, with improvements of between $57\%-77\%$ over our best performing baselines.

Figures (4)

• Figure 1: A schematic of our framework for steering LLMs toward data-driven personas. The bottom-half illustrates the formation of data-driven personas, and the top-half illustrates LLM steering. A persona is defined by generating individual embeddings via collaborative filtering. The persona can be a single individual embedding (grey dots) or the centroid of a group of embeddings, referred to as a cluster persona (denoted by the circled clusters). To steer the LLM we pass an embedding to a soft-prompting model (SPM), which maps the embedding to a set of persona-specific virtual tokens. Finally we prepend these virtual tokens to the tokenized input sequence and pass this into the LLM to obtain a persona-specific response.
• Figure 2: The Political Party, Race and Education composition (from top to bottom) of clusters and overall population.
• Figure 3: The demographic composition of Clusters-0 to 5 and the Overall Population. From left to right and top to bottom, we show demographic composition for Ideology, Region, Age, Citizenship, Marital status, Religion, Sex, Religion attendance and Income.
• Figure 4: Example prompts of baseline methods. The left panel shows an example prompt used for the Demographics + Raw Q. baseline while the right panel shows an example prompt used for the Context + Raw Q. baseline.