Narrowing Action Choices with AI Improves Human Sequential Decisions

TL;DR

This work tackles how to achieve human-AI complementarity in sequential decision making without demanding user expertise to arbitrate when to defer to an AI. It designs a decision support system that constrains human actions to an AI-informed action set, with a controllable agency parameter $\varepsilon$ and stochasticity through a half-normal variable $W$, and formalizes this within a structural causal model. A Lipschitz bandit framework is developed to efficiently identify the optimal $\varepsilon$, leveraging proven Lipschitz properties of the value function and action-set distributions, with a zooming algorithm that attains sublinear simple regret. Empirically, a large-scale wildfire mitigation study shows that system-assisted humans outperform unaided humans by about $29.7\%$, and even surpass the standalone AI agent by around $2.3\%$, validating the method’s practical impact and providing open-source data and code for replication and extension.

Abstract

Recent work has shown that, in classification tasks, it is possible to design decision support systems that do not require human experts to understand when to cede agency to a classifier or when to exercise their own agency to achieve complementarity$\unicode{x2014}$experts using these systems make more accurate predictions than those made by the experts or the classifier alone. The key principle underpinning these systems reduces to adaptively controlling the level of human agency, by design. Can we use the same principle to achieve complementarity in sequential decision making tasks? In this paper, we answer this question affirmatively. We develop a decision support system that uses a pre-trained AI agent to narrow down the set of actions a human can take to a subset, and then asks the human to take an action from this action set. Along the way, we also introduce a bandit algorithm that leverages the smoothness properties of the action sets provided by our system to efficiently optimize the level of human agency. To evaluate our decision support system, we conduct a large-scale human subject study ($n = 1{,}600$) where participants play a wildfire mitigation game. We find that participants who play the game supported by our system outperform those who play on their own by $\sim$$30$% and the AI agent used by our system by $>$$2$%, even though the AI agent largely outperforms participants playing without support. We have made available the data gathered in our human subject study as well as an open source implementation of our system at https://github.com/Networks-Learning/narrowing-action-choices .

Figures (10)

• Figure 1: A causal model of human sequential decision making with action sets. Squares represent endogenous random variables and circles represent exogenous noise variables. An arrow from (to) one variable indicates that it is the cause (effect) of another. The values of all exogenous variables are sampled from fixed (unknown) distributions, while the value of each endogenous variable is given by a function of the values of its ancestors in the causal graph.
• Figure 2: Instance of our wildfire mitigation game as shown to human participants. Tiles with green trees are healthy, tiles with fire icons are burning, and brown tiles are burnt. The size of the fire icon in each burning tile is proportional to the time steps left until the tile turns to burnt on its own. Non-faded burning tiles correspond to the action set given by our decision support system, while faded tiles correspond to actions that the decision support system does not allow the human participant to select.
• Figure 3: Discounted cumulative reward achieved by human participants supported by our system, human participants on their own, and the AI agent used by our system. In the blue line, each point correspond to a different value of the parameter $\varepsilon$ controlling the level of human agency and error bars correspond to $95\%$ confidence intervals. Here, we use $\gamma=0.99$ and, to improve visibility, in the range of $\varepsilon\in[0,0.3]$, we create coarse-grained bins of $\varepsilon$ of size $5$ and, for each bin, we show the average discounted cumulative reward.
• Figure 4: Distribution of discounted cumulative rewards achieved by human participants supported by our system, human participants on their own, and the AI agent used by our system. Within the plot, vertical lines correspond to the respective empirical average discounted cumulative reward. Here, our decision support system uses $\varepsilon \in [0.08, 0.12]$, and a t-test shows that its difference from the respective distribution of the AI agent alone and the average human alone is statistically significant with p-value $0.01$.
• Figure 5: Simple regret achieved by our Lipschitz best-arm identification algorithm against a uniform discretization baseline. Each point in the green (orange) line shows the empirical average simple regret achieved over $100$ executions of Algorithm \ref{['algo:best-arm-identification']} (\ref{['algo:uniform']}), respectively, given the respective exploration budget $n$. Here, we repeat each experiment $100$ times using a different random seed for the payoffs obtained from arm pulls and shaded areas represent $95\%$ confidence intervals. In Algorithm \ref{['algo:best-arm-identification']}, we set $\beta=2$ and $L=150$.

Theorems & Definitions (3)

• Proposition 1
• Proposition 2
• Proposition 3