Language Agents as Digital Representatives in Collective Decision-Making

TL;DR

The paper addresses representing individual human preferences in collective decision-making through digital proxies. It formalizes the framework with a social choice function $h$, a mediation mechanism $\tau$, outcome mapping $f$, and participant utilities $g_i$, and introduces a value-based notion of representativity via the Bellman operator $\mathbb{B}_{\pi,\tau}$ and value function $Q_{\pi,\tau}^t$ to compare true and proxy policies. It proposes three levels of equivalence—conditional cloning, transition-level equivalence, and trajectory-level equivalence—and argues that trajectory-level, value-aware equivalence best captures representativity. Through a consensus-finding case study using a large UK-demographic dataset and fine-tuned 1B and 30B adaptations of the Chinchilla LLM, the authors show that larger, fine-tuned digital representatives can closely reproduce ground-truth critiques and preserve consensus payoffs. The work demonstrates the feasibility of scalable simulations for mechanism design and policy deliberation while emphasizing the need for human validation and careful interpretation of simulators for real-world deployment.

Abstract

Consider the process of collective decision-making, in which a group of individuals interactively select a preferred outcome from among a universe of alternatives. In this context, "representation" is the activity of making an individual's preferences present in the process via participation by a proxy agent -- i.e. their "representative". To this end, learned models of human behavior have the potential to fill this role, with practical implications for multi-agent scenario studies and mechanism design. In this work, we investigate the possibility of training \textit{language agents} to behave in the capacity of representatives of human agents, appropriately expressing the preferences of those individuals whom they stand for. First, we formalize the setting of \textit{collective decision-making} -- as the episodic process of interaction between a group of agents and a decision mechanism. On this basis, we then formalize the problem of \textit{digital representation} -- as the simulation of an agent's behavior to yield equivalent outcomes from the mechanism. Finally, we conduct an empirical case study in the setting of \textit{consensus-finding} among diverse humans, and demonstrate the feasibility of fine-tuning large language models to act as digital representatives.

Figures (6)

• Figure 1: Consensus-Finding. Observe that this is an instance of a collective decision-making setting:
• Figure 2: Critique Evaluation. Left: Mean log-likelihood of ground-truth critiques from human participants (from the validation set), evaluated under models $\hat{\pi}_{i}$ with fine-tuned ("FT'd") or vanilla digital representatives ("DRs") with 1B or 30B parameters. The fine-tuned models consistently exhibit higher log-likelihoods compared to their vanilla counterparts, indicating a superior representation of participants' ground-truth critiques. Right: The autorater's win-rate of a sampled critique (for different models $\hat{\pi}_{i}$ across the x-axis) against one's own ground-truth critique (i.e. the baseline). The golden bar represents $\pi^{*}$ against itself, serving as a reference for ceiling performance. Conclusion: Fine-tuning and scale both improve the representativeness of DRs for held-out participants' critiques.
• Figure 3: Consensus Evaluation. Either one participant's critique (top), or all participants' critiques (bottom) are substituted with critiques sampled from their respective digital representatives $\hat{\pi}_{i}$. Left: Mean discrepancy in payoffs between the revised consensus generated by the mediator mechanism using participants' ground-truth critiques, versus using critiques sampled from DRs. Replacing either one or all ground truth critiques with vanilla DR samples results in significantly degraded payoffs, whereas fine-tuned DR samples yield payoffs with little or no degradation, indicating that those consensuses are more or less equivalent. Right: The autorater's win-rate of a generated revised consensus (using critiques sampled from different models $\hat{\pi}_{i}$) against the revised consensus generated using ground-truth critiques (i.e. the baseline). The golden bar represents the baseline consensus against itself, serving as a reference for ceiling performance. Substituting a single critique (chosen at random) results in consensuses perceived as roughly equally similar across all DRs by the autorater (13% difference between ceiling and Vanilla 1B DRs). This is likely a property of the mediation mechanism, which we empirically observe disregards outlier critiques. However, substituting the entire group's critiques significantly influences the revised consensus output (30% difference between ceiling and Vanilla 1B DRs), with the 30B fine-tuned DRs having a notably higher win-rate here. Conclusion: Using the notion of representativity motivated earlier, in both top and bottom panels we also observe that fine-tuning and scale both improve the representativity of DRs for held-out episodes.
• Figure 4: Variants of Digital Representatives. Mean log-likelihood of ground-truth critiques from human participants (from the validation set), evaluated under various digital representatives. All these DRs have 1B parameters and were fine-tuned on datasets conditioned on diverse additional information, as indicated on the x-axis (refer to Table \ref{['tab:prompt-info']} for details and examples). The optimal variant (Base+O+C) incorporates participant $i$'s opinions and critiques to other questions. This variant performs very similarly to its counterpart that additionally includes demographic information (Base+D+O+C). However, we opted for the former for simplicity. Note that variants based solely on demographics (Base+D or position scoring (Base+P) perform much worse. This suggests that integrating participant-specific few-shot information enhances both task- and self-consistency.
• Figure 5: Autorater Critique Evaluations: Ablations. The autorater's win-rate of sampled critiques against a custom baseline, as specified by the grey bar within each plot (included as a sanity check, expected around 50%). The ground truth (One's Own Critique) is also included to set the performance ceiling. The main take-away is that the 30B fine-tuned DR (green) closely matches the performance of ground truth critiques (golden), suggesting its ability to learn both task-specific aspects (e.g., critique-ness, topic) and participant-specific nuances (e.g., individual preferences, style). A similar trend is observed for the 1B fine-tuned DR, albeit to a lesser extent.

Theorems & Definitions (4)

• Definition 1: restate=defsocial,name=Social Choice Function
• Definition 2: restate=defmechanism,name=Decision Mechanism
• Definition 3: restate=defequivalence,name=Representational Equivalence
• Proposition 1: restate=thmequivalence,name=Representational Equivalence