Intuitions of Compromise: Utilitarianism vs. Contractualism

TL;DR

The paper questions which value-aggregation rule yields intuitively plausible compromises when groups differ in what they value, by contrasting the Utilitarian Sum with the Nash Product under a broad, systematically generated set of scenarios. It introduces precise visual aids (area and volume charts) to communicate the proposals and conducts parallel assessments with humans and GPT-family LLMs, revealing a robust human preference for the Nash Product in disagreement cases and varying alignment in agreement cases. The results challenge the default utilitarian approach in AI alignment and decision-support contexts and suggest contractualist intuitions may better capture lay intuitions about fair trade-offs, especially when accompanied by effective visual communication. The work also highlights limitations and variability across LLMs, underscoring the need for careful design of AI-assisted value aggregation tools and further exploration of contractualist mechanisms beyond the Nash Product.

Abstract

What is the best compromise in a situation where different people value different things? The most commonly accepted method for answering this question -- in fields across the behavioral and social sciences, decision theory, philosophy, and artificial intelligence development -- is simply to add up utilities associated with the different options and pick the solution with the largest sum. This ``utilitarian'' approach seems like the obvious, theory-neutral way of approaching the problem. But there is an important, though often-ignored, alternative: a ``contractualist'' approach, which advocates for an agreement-driven method of deciding. Remarkably, no research has presented empirical evidence directly comparing the intuitive plausibility of these two approaches. In this paper, we systematically explore the proposals suggested by each algorithm (the ``Utilitarian Sum'' and the contractualist ''Nash Product''), using a paradigm that applies those algorithms to aggregating preferences across groups in a social decision-making context. While the dominant approach to value aggregation up to now has been utilitarian, we find that people strongly prefer the aggregations recommended by the contractualist algorithm. Finally, we compare the judgments of large language models (LLMs) to that of our (human) participants, finding important misalignment between model and human preferences.

Figures (21)

• Figure 1: A: An example scenario. We asked participants to choose between three proposals which would differentially effect three equally-sized groups. In this case, each proposal decreases the average cost of a medical visit. We either showed participants just the text on the left (none of the charts) or some combination of charts (area, volume, or both) to aid understanding of the scenarios. We kept the color of the proposals the same in both charts. B: A 3-d, volume chart of the scenario depicted in panel A. Each of the lines labelled "apple", "bee", and "cow" is an axis for each group. The colored boxes "one", "two", and "three" represent the different proposals. Each proposal spans a length on each axis proportional to the outcome for that group. (E.g. The blue box, "one" spans 1 on the "apple" axis, 51 on the "bee" axis, and 51 on the "cow" axis.) These 3-d charts could be dragged around with a cursor to see the boxes from different sides. We tested for this behavior and extensively familiarized participants with these 3-d charts in a qualification task (reproduced in SI § "Qualification Task"). C: A stacked, area chart of the scenario depicted in panel A. Each group appears on the x-axis. The colored bars show the outcome for each proposal for each group.
• Figure 2: The percent of A human (Mturk) participants and Bgpt-4 responses that endorse each value aggregation algorithm: the Utilitarian Sum (an additive model, shown in green with the $\Sigma$ symbol) and the Nash Product (a multiplicative model, shown in orange with the $\Pi$ symbol) on cases in which the two mechanisms disagree. The panels represent the different decision-aids that participants received: area, volume, both, and none. (N=102 per condition.) The dashed line at 33% indicates random guessing. (Participants always selected from three options.) Error bars show 95% binomial confidence intervals. See SI Fig. A.4 for tests with additional LLMs.
• Figure 3: The count of aligned human participants and each of the Nash Product ($\Pi$) and the Utilitarian Sum ($\Sigma$) when those mechanisms disagreed with each other and when they agreed. (See Fig. \ref{['fig:test-conditions']} and \ref{['fig:mturk-control']}.) Columns show the visual aids participants received: the area chart, volume chart, both, or none. (N=102 per cell.) The disagreement cases contained 18 unique scenarios presented with 4 different contexts each answered by 3 unique participants for 216 responses total ($18 \times 4 \times 3$). Similarly, the agreement cases had 132 responses ($11 \times 4 \times 3$). In each case, we run a binomial test with a null hypothesis of random guessing (1/3). $\text{***}: p < .001$; $\text{*}: p < .05$
• Figure 4: The percent of A human participants (Mturk) and Bgpt-4 responses that were aligned with the Utilitarian Sum and the Nash Product on cases in which the two mechanisms agree. The panels represent the decision-aids participants received: area, volume, both, and none. (N=102 per condition.) The dashed line at 33% indicates random guessing. A: High agreement with the Utilitarian Sum and the Nash Product when both agree indicates that the two capture what participants intuit by a "best compromise." B: In comparison to the human results, the lower agreement of LLMs with the Utilitarian Sum and the Nash Product when both agree indicates that computations besides those mechanisms drive the choice of a "best compromise." This figure displays results for gpt-4; see SI Fig. A.5 for tests with additional LLMs.
• Figure 5: The percent agreement between human participants and two aggregation mechanisms: the Nash Product (Eq. \ref{['equation-product']}) and the Inequality Sum (Eq. \ref{['equation-inequality-aversion']}, a variant of the Utilitarian Sum with a term to avoid inequality). The x-axis varies the inequality aversion parameter of the Inequality Sum: from no inequality aversion ($\alpha$=0; equivalent to the Utilitarian Sum) to only inequality aversion, ignoring aggregate utility ($\alpha$=1; similar to the Rawlsian Minimum, Eq. \ref{['eqn:leximin']}). The top, green line () shows the proportion of participants who select the "correct" answer when the Nash Product and Inequality Sum agree on which proposal is best. The other lines track responses in the cases where the two mechanisms disagree. The middle, orange line (o) represents the proportion of subjects who endorsed the proposal consistent with the Nash Product and the bottom, blue, line (+) represents proportion of subjects who endorsed the proposals consistent with the Inequality Sum. There were two points in which there were no disagreements between the Inequality Sum and the Nash Product, suggesting that the two mechanisms may be equivalent here. (N=102 overlapping participants for each point.) Error bars show 95% binomial confidence intervals. (See SI Fig. A.11 for the data of this plot.)