EigenBench: A Comparative Behavioral Measure of Value Alignment

TL;DR

This work collects human judgments on the same ensemble of models and shows that EigenBench's judgments align closely with those of human evaluators, supporting its viability as a framework for evaluating subjective values for which no ground truths exist.

Abstract

Aligning AI with human values is a pressing unsolved problem. To address the lack of quantitative metrics for value alignment, we propose EigenBench: a black-box method for comparatively benchmarking language models' values. Given an ensemble of models, a constitution describing a value system, and a dataset of scenarios, our method returns a vector of scores quantifying each model's alignment to the given constitution. To produce these scores, each model judges the outputs of other models across many scenarios, and these judgments are aggregated with EigenTrust (Kamvar et al., 2003), yielding scores that reflect a weighted consensus judgment of the whole ensemble. EigenBench uses no ground truth labels, as it is designed to quantify subjective traits for which reasonable judges may disagree on the correct label. Hence, to validate our method, we collect human judgments on the same ensemble of models and show that EigenBench's judgments align closely with those of human evaluators. We further demonstrate that EigenBench can recover model rankings on the GPQA benchmark without access to objective labels, supporting its viability as a framework for evaluating subjective values for which no ground truths exist. The code is available at https://github.com/jchang153/EigenBench.

Figures (10)

• Figure 1: The EigenBench Pipeline: Starting with a population of models $\mathcal{M} = \{M_1,\ldots,M_N\}$, a constitution $\mathcal{C}$, and a set of prompted scenarios $\mathcal{S}$, we repeatedly sample a scenario $S_\ell \in \mathcal{S}$, prompt a pair of models $M_j, M_k$ with the scenario, prompt a third model $M_i$ to judge which response is more aligned to $\mathcal{C}$, fit the resulting judgments $r_{ijkl}$ to a Bradley-Terry-Davidson model of pairwise preferences to learn model dispositions and judge lenses in a latent space $\mathbb{R}^d$, derive a trust matrix indicating how often judge $M_i$ favors evaluee $M_j$'s responses, extract the left eigenvector $\mathbf{t}$ of the trust matrix, and convert $\mathbf{t}$ to Elo ratings that indicate, in the aggregate judgment of the population $\mathcal{M}$, each model's degree of alignment to $\mathcal{C}$. Importantly, only the judge receives the constitution; the evaluees do not know what criteria will be used to evaluate their responses (or even that they will be evaluated at all).
• Figure 2: Learned model dispositions $v_j$ and judge lenses $u_i$ in a $2$-dimensional latent space for Claude 3.5 Haiku prompted with $20$ different historical personas on the Universal Kindness constitution $\mathcal{C}$. Left: each triangle represents a judge lens $u_i \in \mathbb{R}^2$, sized inversely proportional to its tie propensity $\lambda_i$. All learned tie propensities are in the interval $[1.15, 1.62]$. Right: each circle represents a model disposition $v_j \in \mathbb{R}^2$. In our fit Bradley-Terry-Davidson model, the log latent strength of model $j$, as judged by model $i$, is the the inner product $u_i^\top v_j$ of $i$'s judge lens vector with $j$'s disposition vector. All learned judge lenses are in the first quadrant of $\mathbb{R}^2$, so the personas judged most aligned to $\mathcal{C}$ are at the top right of the model dispositions plot (MLK persona was judged the most "kind") and the personas judged least aligned to $\mathcal{C}$ are at the bottom left (Lenin and Nietzsche personas were judged the least "kind"). The learned judge lenses organize along a secular-to-sacred axis (from Feynman and Lenin on the left side to Pope Francis on the right side), indicating a difference in how sacred and secular personas interpret the same constitution.
• Figure 3: EigenBench Elo scores for eight models judged on the Universal Kindness, Conservatism, and Deep Ecology constitutions. The 95% confidence intervals shown are derived from the bootstrapping percentile method efron1994introduction. Larger confidence intervals are apparent in the scores for Deep Ecology due to a large portion of ties in the pairwise comparisons, as fewer scenarios are relevant to the constitution.
• Figure 4: Learned dispositions $v_j$ and judge lenses $u_i$ in a $2$-dimensional latent space, for $5\times 5$ (LM, persona) pairs. Persona prompts and the constitution used (Universal Kindness) can be found in Appendix \ref{['app:constitutions']}. Left: judge lens $u_i\in\mathbb{R}^2$, sized inversely proportional to its tie propensity $\lambda_i$. All learned tie propensities are in the interval $[0.34, 2.27]$. Right: model disposition $v_j \in \mathbb{R}^2$.
• Figure 5: EigenBench trust scores for a population of 5 LMs x 5 Personas on the Universal Kindness constitution. For example, the kindest combination as judged by these $25$ models is Gemini 2.5 Pro with the Empathetic prompted persona. 21% of the variance in these trust scores is explained by the LM and 79% of the variance is explained by the persona.