Plurals: A System for Guiding LLMs Via Simulated Social Ensembles

TL;DR

Plurals introduces a general-purpose system for pluralistic AI by enlisting multiple LLM agents to deliberate under configurable structures and moderators. It grounds design in deliberative democracy and pluralistic sociotechnical systems, and leverages ANES-based nationally representative personas to diversify viewpoints. Six case studies show fidelity to theoretical constructs and that simulated focus groups yield outputs resonant with target audiences, outperforming zero-shot baselines in multiple trials. The open-source Python package provides a modular platform for studying multi-agent AI dynamics, facilitating responsible development and exploration of AI guardrails and alignment in human-centric contexts.

Abstract

Recent debates raised concerns that language models may favor certain viewpoints. But what if the solution is not to aim for a 'view from nowhere' but rather to leverage different viewpoints? We introduce Plurals, a system and Python library for pluralistic AI deliberation. Plurals consists of Agents (LLMs, optionally with personas) which deliberate within customizable Structures, with Moderators overseeing deliberation. Plurals is a generator of simulated social ensembles. Plurals integrates with government datasets to create nationally representative personas, includes deliberation templates inspired by deliberative democracy, and allows users to customize both information-sharing structures and deliberation behavior within Structures. Six case studies demonstrate fidelity to theoretical constructs and efficacy. Three randomized experiments show simulated focus groups produced output resonant with an online sample of the relevant audiences (chosen over zero-shot generation in 75% of trials). Plurals is both a paradigm and a concrete system for pluralistic AI. The Plurals library is available at https://github.com/josh-ashkinaze/plurals and will be continually updated.

Figures (5)

• Figure 1: System diagram of Plurals---an end-to-end generator of simulated social ensembles. (1) Agents complete tasks within (2) Structures, with communication optionally summarized by (3) Moderators. Plurals integrates with government datasets (1a) and templates inspired by deliberative democracy theory (1b). The building block is Agents, which are large language models (LLMs) that have system instructions and tasks. System instructions can be generated from user input, government datasets (American National Election Studies; ANES), and templates from deliberative democracy literature bachtiger_deliberative_2018. Agents exist within Structures, which define what information is shared. Combination instructions tell Agents how to combine the responses of other Agents when deliberating in the Structure. Users can customize an Agent's combination instructions or use existing templates drawn from deliberation literature and beyond. Moderators aggregate responses from multi-agent deliberation.
• Figure 2: Plurals allows users to create complex and customizable deliberations with a few lines of intuitive code. These code snippets are annotated with the features they display. For up-to-date syntax and snippets, see the GitHub repository and associated documentation.
• Figure 3: Current Structures that Plurals supports: Chain, Graph, Debate, and Ensemble. A Chain is a sequence of agents arranged in a customizable order, with the option to shuffle the order on each cycle. A Graph is a directed acyclic graph of agents where users provide agents and edges, enabling deliberation to proceed through the graph where ($A \rightarrow B$) implies B will see A's responses. Debate involves exactly two agents engaging in back-and-forth discussions. An Ensemble is a list of agents processing tasks in parallel. Plurals also supports the creation of custom structures (Appendix \ref{['custom_structures']}).
• Figure 4: In three experiments, both zero-shot and Plurals simulated focus groups tried to create output compelling to specific audiences. Plurals simulated focus group output was chosen by an online sample of the relevant audiences over zero-shot. See SM Table 1 for multilevel regressions.
• Figure 5: Comparison of lexical diversity metrics for GPT-4o and Claude Sonnet. Each dot is one corpus evaluated for a given metric. Higher values indicate more diversity; Red dots are Plurals ANES personas and blue dots are non-Plurals, ideology-only personas. For 95% of GPT-4o corpora, and 100% of Claude Sonnet corpora, Plurals personas (red) have higher lexical diversity than non-Plurals prompting (blue). TTR is the ratio of unique n-grams to total n-grams. HD-D applies an adjustment for varying word lengths to TTR.