Effect of Adaptation Rate and Cost Display in a Human-AI Interaction Game

TL;DR

This work investigated how changes in the AI's adaptive algorithm impact behavior predictions in two-player continuous games and demonstrated that AI adaptation rate can significantly affect human behavior, having the ability to shift the outcome between two game theoretic equilibrium.

Abstract

As interactions between humans and AI become more prevalent, it is critical to have better predictors of human behavior in these interactions. We investigated how changes in the AI's adaptive algorithm impact behavior predictions in two-player continuous games. In our experiments, the AI adapted its actions using a gradient descent algorithm under different adaptation rates while human participants were provided cost feedback. The cost feedback was provided by one of two types of visual displays: (a) cost at the current joint action vector, or (b) cost in a local neighborhood of the current joint action vector. Our results demonstrate that AI adaptation rate can significantly affect human behavior, having the ability to shift the outcome between two game theoretic equilibrium. We observed that slow adaptation rates shift the outcome towards the Nash equilibrium, while fast rates shift the outcome towards the human-led Stackelberg equilibrium. The addition of localized cost information had the effect of shifting outcomes towards Nash, compared to the outcomes from cost information at only the current joint action vector. Future work will investigate other effects that influence the convergence of gradient descent games.

Figures (8)

• Figure 1: Human participants played a continuous game with an adaptive AI where they were shown one of two display interfaces: (a) current human cost (Experiment 1 annotated screenshot), similar to seeing a single point on their cost landscape, and (b) localized heat map (Experiment 2 annotated screenshot), similar to seeing a localized subsection of their perceived cost landscape.
• Figure 2: Results of Experiment 1 (cost feedback) with $30$ participants. The median actions and costs are compared to the Nash and Stackelberg equilibria. Experiment 1 shows increasing the AI's adaptation rates shifts the outcome from the Nash to the Stackelberg equilibrium.
• Figure 3: Results of Experiment 2 (cost landscape feedback) with 30 participants. The median actions and costs are compared to the Nash and Stackelberg equilibria. Experiment 2, when compared to Experiment 1, shows a further shift towards the Nash equilibrium at slower adaptation rates.
• Figure 4: Comparing last 5 seconds of $H$'s actions with ticks displaying analytically calculated equilibria and median actions (Experiment 1 vs Experiment 2). Experiment 2 shows a shifted distribution of actions towards the Nash equilibrium.
• Figure 5: Results of simulating the learning dynamics of a game with action vector dimensions of 64x128. For slow adaptation rates, the actions shift towards the Nash equilibrium, while for fast adaptation rates, the actions shift towards the Stackelberg equilibrium, matching the experiments.