Hybrid Human-Agent Social Dilemmas in Energy Markets

Abstract

In hybrid populations where humans delegate strategic decision-making to autonomous agents, understanding when and how cooperative behaviors can emerge remains a key challenge. We study this problem in the context of energy load management: consumer agents schedule their appliance use under demand-dependent pricing. This structure can create a social dilemma where everybody would benefit from coordination, but in equilibrium agents often choose to incur the congestion costs that cooperative turn-taking would avoid. To address the problem of coordination, we introduce artificial agents that use globally observable signals to increase coordination. Using evolutionary dynamics, and reinforcement learning experiments, we show that artificial agents can shift the learning dynamics to favour coordination outcomes. An often neglected problem is partial adoption: what happens when the technology of artificial agents is in the early adoption stages? We analyze mixed populations of adopters and non-adopters, demonstrating that unilateral entry is feasible: adopters are not structurally penalized, and partial adoption can still improve aggregate outcomes. However, in some parameter regimes, non-adopters may benefit disproportionately from the cooperation induced by adopters. This asymmetry, while not precluding beneficial entry, warrants consideration in deployment, and highlights strategic issues around the adoption of AI technology in multiagent settings.

Figures (6)

• Figure 1: Centralized vs Decentralized outcomes for 130 testing days across different population sizes. Selfish agents learn subobtimal policies.
• Figure 2: All possible equilibrium payoffs of the repeated games as the inconvenience cost $p$ varies. The shaded region illustrates mutually beneficial equilibrium payoffs; larger inconvenience costs shrink the space of cooperative equilibria.
• Figure 3: Convergence of $1000$ simulations of replicator dynamics with $p =0.5$ for two populations P1 and P2, comparing low versus high patience regimes. (a) For $\delta = 0.51$, the system predominantly converges to mixtures of PPP and PPA, resulting in non-cooperative outcomes. (b) For $\delta = 0.95$, a greater proportion of simulations converge to the cooperative turn-taking equilibria (PPA, APA).
• Figure 4: Average trajectory of system cost with intrinsic reward terms in the full DSLM/RL environment. (a) Basic scenario: agents adapt by alternating their actions, demonstrating turn-taking. (b) 4-appliance scenario: the intrinsic reward bonus enables agents to more frequently identify the socially optimal Nash equilibrium.
• Figure 5: Convergence of $1000$ simulations of replicator dynamics with $p =0.5$ and intrinsic reward terms ($\Omega = 100$) for two populations P1 and P2. (a) For $\delta = 0.51$, the system converges to cooperative equilibria, in contrast to the non-cooperative outcome without intrinsic rewards (Figure \ref{['fig:replicator-hist']}a). (b) For $\delta = 0.95$, cooperative convergence is similarly achieved.