Inverse decision-making using neural amortized Bayesian actors

TL;DR

This work addresses inverse decision-making for Bayesian actor models in continuous-action tasks by amortizing the forward decision problem with a neural surrogate trained in an unsupervised fashion on the decision-cost objective. The surrogate enables efficient gradient-based Bayesian inference over the actor's parameters, including priors, sensory noise, and costs. Through synthetic benchmarks and three real sensorimotor datasets, the method achieves near-analytical accuracy when available, facilitates model comparison, and reveals identifiability limits between priors and costs while offering strategies to disentangle them via experimental design. The approach broadens the applicability of normative Bayesian accounts to naturalistic, continuous-action behavior and provides a practical, scalable tool for estimating internal decision factors from observed behavior.

Abstract

Bayesian observer and actor models have provided normative explanations for many behavioral phenomena in perception, sensorimotor control, and other areas of cognitive science and neuroscience. They attribute behavioral variability and biases to interpretable entities such as perceptual and motor uncertainty, prior beliefs, and behavioral costs. However, when extending these models to more naturalistic tasks with continuous actions, solving the Bayesian decision-making problem is often analytically intractable. Inverse decision-making, i.e. performing inference over the parameters of such models given behavioral data, is computationally even more difficult. Therefore, researchers typically constrain their models to easily tractable components, such as Gaussian distributions or quadratic cost functions, or resort to numerical approximations. To overcome these limitations, we amortize the Bayesian actor using a neural network trained on a wide range of parameter settings in an unsupervised fashion. Using the pre-trained neural network enables performing efficient gradient-based Bayesian inference of the Bayesian actor model's parameters. We show on synthetic data that the inferred posterior distributions are in close alignment with those obtained using analytical solutions where they exist. Where no analytical solution is available, we recover posterior distributions close to the ground truth. We then show how our method allows for principled model comparison and how it can be used to disentangle factors that may lead to unidentifiabilities between priors and costs. Finally, we apply our method to empirical data from three sensorimotor tasks and compare model fits with different cost functions to show that it can explain individuals' behavioral patterns.

Figures (9)

• Figure 1: A Bayesian decision-making problem from the perspective of the subject. The subject needs to find the optimal action $a^*$ based on the sensory measurement $m$ of the state of the world $s$. Combined with a cost function $\ell(r, s)$ and the action distribution $r \sim \mathop{\mathrm{\text{Lognormal}}}\nolimits(a, \sigma_r)$, they want to minimize a cost function under their belief about the state of the world which yields the optimal action $a^* = \mathop{\mathrm{arg\,min}}\limits_a \mathbb{E}_{p(s \:\vert\:m)} \left[ \mathbb{E}_{p(r \:\vert\:a)} \left[ \ell(r, s) \right] \right]$. B Bayesian inference problem about the subject's parameters from the perspective of the researcher. The researcher solves the inverse decision-making problem, i.e. they want to infer the posterior distribution $p({\bm\theta} \mid \mathcal{D})$ over the parameters ${\bm\theta}$ of the subject's perception-action system and cost function given a dataset $\mathcal{D} = \{s_i, r_i: i = 1, \dots, n\}$ of stimuli $s_i$ and responses $r_i$ from $n$ trials. To make inference of the posterior over ${\bm\theta}$ feasible, we use a neural network as an approximator for the optimal action $a^*$.
• Figure 2: A Simulated data from the Bayesian actor model with quadratic cost function are shown as a scatter plot. The posterior predictive distribution $p(r_\text{pred} \mid s, r)$ (mean and 94% CI) obtained using our method is shown as a blue line with shaded region. B Posterior distributions of parameters obtained using the analytical solution or the neural network to compute optimal actions. The top plot of each column shows the respective marginal posterior distribution for each parameter. Ground truth parameter values are shown for comparison. C Evaluation (MSE between posterior mean and ground truth) for multiple runs with uniformly sampled ground truth parameters.
• Figure 3: A Simulated behavior from the asymmetric quadratic cost (\ref{['eq:aqc']}). The responses exhibit undershots, which could be due to the prior or due to the cost. B Posteriors distributions: for each pair of parameters, the plot shows the contours of the 94%-HDI. The pairwise posterior between prior mean $\mu_0$ and cost asymmetry parameter $\alpha$ shows a strong correlation. C MSE between ground truth and posterior mean when fixing no parameters, the cost asymmetry parameter $\alpha$ or the prior parameter $\mu_0$. Fixing one of the confounding variables results in an improvement of accuracy in the inference of the other variable. The remaining parameters maintain their accuracy.
• Figure 4: Disentangling priors and costs using different levels of perceptual uncertainty. A Data simulated with two different levels of perceptual uncertainty $\sigma$. B Inferred posterior distributions (94% CIs) for an experiment with 45 trials of each of two different levels of perceptual uncertainty (purple) and with 90 trials of one level of perceptual uncertainty (shades of blue).
• Figure 5: Data of exemplary subjects from three different tasks. The tasks were throwing bean bags at a target willey2018long, sliding a puck to a target neupartl2020intuitive and producing a force of certain magnitude onneweer2016force, from left to right. Rows show different aspects of the data. A Data of the subject with mean and 94% confidence interval of posterior predictive distributions along with a qualitative plot of the cost function with the inferred mean parameters (thick) and the inferred 94%-HDI bounds (light background) over the response $r$ with fixed stimulus $s$ in the upper left panel. B Inferred means and 94%-CI for prior mean $\mu_0$ for a range of different cost functions, grouped by their functional form as symmetric, symmetric plus effort term or asymmetric. The cost functions are given explicitly in \ref{['app:tab:cost_functions']}. C Model comparison based on the differences in PSIS-LOO estimates of the expected log point-wise predictive density (ELPD) of different cost functions to the best scoring cost function (lower is better). Error bars denote standard error.