Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Safe Pareto Improvements for Expected Utility Maximizers in Program Games

Anthony DiGiovanni, Jesse Clifton, Nicolas Macé

TL;DR

This paper tackles miscoordination in bargaining-like interactions among agents who can condition actions on others' code. It develops Safe Pareto improvements ($SPIs$) in program games by constructing renegotiation-based SPIs and proving that, under mild belief assumptions, agents prefer renegotiation and can guarantee at least the Pareto meet minimum ($PMM$) payoff; it also shows that the PMM bound is tight and renegotiation alone cannot guarantee improvements beyond PMM. To address the SPI selection problem, the authors introduce conditional set-valued renegotiation (CSR), which uses intersecting renegotiation sets and a selection function to produce Pareto-efficient agreements that preserve PMM guarantees. The work combines program equilibrium, renegotiation concepts, and CSR to provide a partial solution for coordinating SPIs in settings where agents may miscoordinate, with potential implications for cooperative AI and policy design. The results offer a principled framework for designing conditional commitment mechanisms that are robust to miscoordination in strategic interactions.

Abstract

Agents in mixed-motive coordination problems such as Chicken may fail to coordinate on a Pareto-efficient outcome. Safe Pareto improvements (SPIs) were originally proposed to mitigate miscoordination in cases where players lack probabilistic beliefs as to how their delegates will play a game; delegates are instructed to behave so as to guarantee a Pareto improvement on how they would play by default. More generally, SPIs may be defined as transformations of strategy profiles such that all players are necessarily better off under the transformed profile. In this work, we investigate the extent to which SPIs can reduce downsides of miscoordination between expected utility-maximizing agents. We consider games in which players submit computer programs that can condition their decisions on each other's code, and use this property to construct SPIs using programs capable of renegotiation. We first show that under mild conditions on players' beliefs, each player always prefers to use renegotiation. Next, we show that under similar assumptions, each player always prefers to be willing to renegotiate at least to the point at which they receive the lowest payoff they can attain in any efficient outcome. Thus subjectively optimal play guarantees players at least these payoffs, without the need for coordination on specific Pareto improvements. Lastly, we prove that renegotiation does not guarantee players any improvements on this bound.

Safe Pareto Improvements for Expected Utility Maximizers in Program Games

TL;DR

This paper tackles miscoordination in bargaining-like interactions among agents who can condition actions on others' code. It develops Safe Pareto improvements () in program games by constructing renegotiation-based SPIs and proving that, under mild belief assumptions, agents prefer renegotiation and can guarantee at least the Pareto meet minimum () payoff; it also shows that the PMM bound is tight and renegotiation alone cannot guarantee improvements beyond PMM. To address the SPI selection problem, the authors introduce conditional set-valued renegotiation (CSR), which uses intersecting renegotiation sets and a selection function to produce Pareto-efficient agreements that preserve PMM guarantees. The work combines program equilibrium, renegotiation concepts, and CSR to provide a partial solution for coordinating SPIs in settings where agents may miscoordinate, with potential implications for cooperative AI and policy design. The results offer a principled framework for designing conditional commitment mechanisms that are robust to miscoordination in strategic interactions.

Abstract

Agents in mixed-motive coordination problems such as Chicken may fail to coordinate on a Pareto-efficient outcome. Safe Pareto improvements (SPIs) were originally proposed to mitigate miscoordination in cases where players lack probabilistic beliefs as to how their delegates will play a game; delegates are instructed to behave so as to guarantee a Pareto improvement on how they would play by default. More generally, SPIs may be defined as transformations of strategy profiles such that all players are necessarily better off under the transformed profile. In this work, we investigate the extent to which SPIs can reduce downsides of miscoordination between expected utility-maximizing agents. We consider games in which players submit computer programs that can condition their decisions on each other's code, and use this property to construct SPIs using programs capable of renegotiation. We first show that under mild conditions on players' beliefs, each player always prefers to use renegotiation. Next, we show that under similar assumptions, each player always prefers to be willing to renegotiate at least to the point at which they receive the lowest payoff they can attain in any efficient outcome. Thus subjectively optimal play guarantees players at least these payoffs, without the need for coordination on specific Pareto improvements. Lastly, we prove that renegotiation does not guarantee players any improvements on this bound.
Paper Structure (22 sections, 7 theorems, 4 equations, 4 figures, 2 tables, 3 algorithms)

This paper contains 22 sections, 7 theorems, 4 equations, 4 figures, 2 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Program equilibrium and commitment games.
  4. Coordination problems and equilibrium selection.
  5. Miscoordination and Safe Pareto Improvements in Program Games
  6. Setup: Program Games and Subjective Equilibrium
  7. Constructing Safe Pareto Improvements via Renegotiation
  8. Incentives to Renegotiate
  9. Proof Sketch.
  10. The SPI Selection Problem and Conditional Set-Valued Renegotiation
  11. Overview of CSR
  12. Components of CSR
  13. Set-valued renegotiation
  14. Conditional renegotiation sets.
  15. Renegotiation sets that guarantee the PMM
  16. ...and 7 more sections

Key Result

Proposition 1

Let $\mathbf{rn}$ be a renegotiation function. For $i=1,2$, define $f_i: P_i \rightarrow P_i^{\textnormal{rn}}(\mathbf{rn})$ such that, for each $p_i \in P_i$, $f_i(p_i)$ is of the form given in Algorithm alg:renegotiation with ${f_i(p_i)}^{\textnormal{def}} = p_i$. Then, the function $\textbf{f}: \

Figures (4)

  • Figure 1: Illustration of the Pareto meet projection (PMP) of three different outcomes (black points) in the Scheduling Game, for each player. Gray points represent payoffs at each pure strategy profile. Each black point is mapped via a player's PMP (black arrows) to a set containing a) the "nearest" point in the Pareto meet and b) all points better for the given player and no better for the other player than (a).
  • Figure 2: Set-valued renegotiation in the Scheduling Game, for two possible player 2 renegotiation sets. Black points represent renegotiation outcomes (mapped from the miscoordination outcome $(0,0)$). If player 1 uses the renegotiation set shown here, they can achieve a Pareto improvement even if players don't reach the Pareto frontier (left), while still allowing for their best possible outcome (right).
  • Figure 3: Two possible renegotiation procedures in the Scheduling Game, for different player 2 renegotiation sets. Player 1 might add the PMM (star) to their unconditional renegotiation set. In the case in the left plot, player 1 is no worse off by adding the PMM to their set. But in the case in the right plot, if player 1 adds the PMM, they might do worse if the selection function chooses the PMM instead of the black point that would have otherwise been achieved.
  • Figure 4: Illustration of the argument for Theorem \ref{['prop:yudpoint']}. By default, the renegotiation outcome is the black circle, $\boldsymbol{a}(\textbf{p})$. Player 1 considers whether to add to their renegotiation set $\mathbf{RN}^{1}(\mathbf{RN}^{2}, \boldsymbol{a}({\textbf{p}}^{\textnormal{def}}))$ the black striped segment $\mathbf{PMP}_1(\boldsymbol{a}(\textbf{p}))$. Player 1 is certain that player 2 would not change their set $\mathbf{RN}^{2}(\mathbf{RN}^{1}, \boldsymbol{a}({\textbf{p}}^{\textnormal{def}}))$ in response to this addition in a way that would make player 1 worse off (Assumption \ref{['assum:nopunish']}). This is because the only change player 1 has made is to add outcomes that make both players weakly better off than $\boldsymbol{a}(\textbf{p})$ and do not make player 2 strictly better off.

Theorems & Definitions (19)

  • Definition 1
  • Example 3.1
  • Definition 2
  • Definition 3
  • Proposition 1
  • Example 3.2
  • Proposition 1
  • Definition 5
  • Definition 6
  • Example 4.1
  • ...and 9 more