On the Effect of Time Preferences on the Price of Anarchy

TL;DR

This work investigates how agents' time preferences affect efficiency in a stateless mean-field game where actions congest shared resources. By contrasting exponential and power-law discounting, it shows PoA is infinite under exponential discounting and equals 2 under power-law discounting when stationary equilibria exist, with no discounting yielding the same PoA as the long-run-average case. The authors establish existence and stability results, relate the mean-field game to stable population games, and analyze learning dynamics, including scenarios where discounting creates unstable equilibria. The findings have practical implications for designing scalable congestion-control mechanisms and timing-aware resource allocation in crowdsourcing, networks, and ride-sharing, illustrating how discounting shapes efficiency and dynamics in large populations.

Abstract

This paper examines the impact of agents' myopic optimization on the efficiency of systems comprised by many selfish agents. In contrast to standard congestion games where agents interact in a one-shot fashion, in our model each agent chooses an infinite sequence of actions and maximizes the total reward stream discounted over time under different ways of computing present values. Our model assumes that actions consume common resources that get congested, and the action choice by an agent affects the completion times of actions chosen by other agents, which in turn affects the time rewards are accrued and their discounted value. This is a mean-field game, where an agent's reward depends on the decisions of the other agents through the resulting action completion times. For this type of game we define stationary equilibria, and analyze their existence and price of anarchy (PoA). Overall, we find that the PoA depends entirely on the type of discounting rather than its specific parameters. For exponential discounting, myopic behaviour leads to extreme inefficiency: the PoA is infinity for any value of the discount parameter. For power law discounting, such inefficiency is greatly reduced and the PoA is 2 whenever stationary equilibria exist. This matches the PoA when there is no discounting and players maximize long-run average rewards. Additionally, we observe that exponential discounting may introduce unstable equilibria in learning algorithms, if action completion times are interdependent. In contrast, under no discounting all equilibria are stable.

Figures (4)

• Figure 1: Plots of the dynamics \ref{['dynamic:average']} and \ref{['dynamic:discounted']}. The parameters are $m=2$, $(t_1,t_2)=(3,0.5)$, $(r_1,r_2)=(e^5,e)$, $(\gamma_1,\gamma_2)=(2,1)$, $b=1$ for both dynamics \ref{['dynamic:average']} and \ref{['dynamic:discounted']} and $\beta=1$ for \ref{['dynamic:discounted']}. The direction of the arrow indicates the direction of mass flow: towards action 1 (lower left) or towards action 2 (upper right). The length of the arrows represents the magnitude of the vector $(\dot\mu_1,\dot\mu_2)$ of the projection dynamics, and the rest points are the equilibrium points. Under no discounting, there is a unique and stable equilibrium (the lower right corner) in the left figure. Under discounting, there are three equilibrium points in the right figure. We always keep the same stable equilibrium as in the undiscounted case, but introduce one more stable equilibrium (at the upper left corner) and an unstable equilibrium (in between the two).
• Figure 2: A system where agents choose between two independent actions when they are not busy. For each action $i\in\{1,2\}$, the time that it takes for an agent to execute the action, i.e., the sojourn time of the action $\tau_i$, is the sum of the waiting time in the resource queue $w_i$ plus the time to traverse the delay element that correspond to the intrinsic time of the action $t_i$. $\mu_i$ is the number of agents performing action $i$, and it satisfies Little's Law $\mu_i=x_i\tau_i$ where $x_i$ is the rate of agents performing action $i$. In equilibrium, the rate $x_i$ can not exceed the supply rate $b_i$ of the resource, i.e., $x_i\leq b_i$. Under no discounting, agents should pursue the action with the largest average reward per unit time $\frac{r_i}{\tau_i}$ unless both actions generate the same average reward.
• Figure 3: $n$ actions with constant execution times. The figure displays mass distribution $\boldsymbol{\mu}^\dagger$ at equilibrium. If Assumption \ref{['assum:constant_action_time']} holds, the agents will opt for the first $i$ actions in the equilibrium, i.e., only $\mu^\dagger_1,\cdots,\mu^\dagger_i$ are positive, and queues must have formed for actions $1,\ldots,i-1$, and possibly for $i$. The vertical thick line on the left shows the size of the queue formed for each action. Actions with higher values of $V_i$ are chosen first, creating longer queues that reduce their attractiveness in terms of reward rates
• Figure 4: Viewing a discrete reward process as a continuous reward process from the corresponding step functions below the curves $y=\frac{r_1}{\tau_1}x^{-\alpha}$ and $y=\frac{r_2}{\tau_2}x^{-\alpha}$. This process is similar to approximating the integral of the function $y=\frac{r_i}{\tau_i}x^{-\alpha}$ by the sum of areas of the rectangles below the curve.

Theorems & Definitions (19)

• definition thmcounterdefinition
• theorem thmcountertheorem
• proposition thmcounterproposition
• proposition thmcounterproposition
• corollary thmcountercorollary
• proposition thmcounterproposition
• theorem thmcountertheorem
• corollary thmcountercorollary
• proposition thmcounterproposition
• proposition thmcounterproposition