More Than Irrational: Modeling Belief-Biased Agents

TL;DR

This work introduces a computational-rationality framework for modeling belief-biased agents whose sub-optimal actions arise from memory-bound cognitive biases. It defines an explicit memory process $f_{\theta}$ that corrupts histories and yields biased beliefs $\tilde{b}_t$, enabling the CR agent to act optimally under $\tilde{b}_t$ with policy $\pi_*(\cdot|\tilde{b};\theta)$. To infer latent memory bounds online, the authors develop Nested Particle Filtering (NPF) to jointly track the internal memory state $\tilde{h}_{t-1}$ and the bound $\theta$ from passively observed actions, demonstrating accurate recovery with limited data. They validate the model on a T-maze task and show how the inferred bounds can power adaptive assistive-POMDP behavior, enabling AI assistants to tailor support to memory limitations. This provides a principled path toward modeling and assisting human collaborators whose decisions reflect bounded memory and biased beliefs.

Abstract

Despite the explosive growth of AI and the technologies built upon it, predicting and inferring the sub-optimal behavior of users or human collaborators remains a critical challenge. In many cases, such behaviors are not a result of irrationality, but rather a rational decision made given inherent cognitive bounds and biased beliefs about the world. In this paper, we formally introduce a class of computational-rational (CR) user models for cognitively-bounded agents acting optimally under biased beliefs. The key novelty lies in explicitly modeling how a bounded memory process leads to a dynamically inconsistent and biased belief state and, consequently, sub-optimal sequential decision-making. We address the challenge of identifying the latent user-specific bound and inferring biased belief states from passive observations on the fly. We argue that for our formalized CR model family with an explicit and parameterized cognitive process, this challenge is tractable. To support our claim, we propose an efficient online inference method based on nested particle filtering that simultaneously tracks the user's latent belief state and estimates the unknown cognitive bound from a stream of observed actions. We validate our approach in a representative navigation task using memory decay as an example of a cognitive bound. With simulations, we show that (1) our CR model generates intuitively plausible behaviors corresponding to different levels of memory capacity, and (2) our inference method accurately and efficiently recovers the ground-truth cognitive bounds from limited observations ($\le 100$ steps). We further demonstrate how this approach provides a principled foundation for developing adaptive AI assistants, enabling adaptive assistance that accounts for the user's memory limitations.

Figures (8)

• Figure 1: Rendering of the T-maze simulation task. The agent (red triangle) must explore the bottom room to find the target object (ball in this case), memorize it, and navigate to the corresponding terminal state (one of the arms of the "T", here the right-side) to succeed.
• Figure 2: Representative trajectories generated by our CR model for agents with different memory bound parameters $\theta$. The color of the trajectory indicates the temporal order of the steps (brighter is earlier), and the colored grids indicate the visit frequency. The agent starts in the hallway (green star) and reaches one of the terminal states (red star). (a) With perfect memory, the agent acts optimally by going down to find the object and directly proceeds to the correct terminal state. (b) With a 40% chance of losing memory, the agent learned to collect more observations on the object. (c) With a 70% chance of losing memory, the agent learned to recheck the found position after it appeared to have forgotten. (d) With no memory, the agent learned to take a random guess without exploring the environment.
• Figure 3: Visualization of $p(\theta|h_t)$ evolving over time where $\theta_{\text{true}}=0.4$ and $\tau=3.0$. Each panel shows the estimated posterior at representative timesteps, where the histogram represents the distribution of the weighted particles and the curve is the kernel density estimate used to approximate the posterior. The posterior mean (blue line) and MAP estimate (red line) rapidly converge towards the true value (green line) as more observations are made.
• Figure 4: Inference error convergence: the PM error and MAP error over time (mean $\pm$ standard error over 5 seeds and all $\theta_{\text{true}}$). Relative to the initial error at $t=1$, the PM error decreased by 90% and 95% at $t=45$ and $t=78$ respectively, demonstrating the inference efficiency.
• Figure 5: The learned adaptive assistance policy shows the distribution of assistance types provided by the AI assistant to simulated users with different ground-truth memory decay rates ($\theta$). The AI learned not to intervene for users with good memory ($\theta \le 0.3$), provide timely memory hints for moderately forgetful users, and provide more direct action hints for users with severe memory decay ($\theta \ge 0.9$).