Theory for Optimal Estimation and Control under Resource Limitations and Its Applications to Biological Information Processing and Decision-Making

TL;DR

The paper addresses how resource limitations—finite memory, intrinsic noise, and energy costs—shape biological information processing and decision-making, challenging the sufficiency of Bayesian filtering in such regimes.It develops a resource-limited optimal estimation and control theory in which a finite memory $d_z$, memory dynamics with noise $F$, and memory-control costs are integrated into a unified framework solved via Hamilton-Jacobi-Bellman and Fokker-Planck equations, yielding optimal estimators $\hat{x}^*$ and memory controls $v^*$ (and optionally state controls $u^*$).Applying the theory to minimal models reveals discontinuous phase transitions between memory-less and memory-based strategies as system parameters vary, illustrating how resource constraints can induce qualitative changes in information processing and decision-making.The framework offers a principled lens to interpret biological diversity in computation, with potential extensions to higher-dimensional environments, numerical methods for high-dimensional HJB-FP problems, and applications to collective and multi-agent biological systems.

Abstract

Despite being optimized, the information processing of biological organisms exhibits significant variability in its complexity and capability. One potential source of this diversity is the limitation of resources required for information processing. However, we lack a theoretical framework that comprehends the relationship between biological information processing and resource limitations and integrates it with decision-making conduced downstream of the information processing. In this paper, we propose a novel optimal estimation and control theory that accounts for the resource limitations inherent in biological systems. This theory explicitly formulates the memory that organisms can store and operate and obtains optimal memory dynamics using optimal control theory. This approach takes account of various resource limitations, such as memory capacity, intrinsic noise, and energy cost, and unifies state estimation and control. We apply this theory to minimal models of biological information processing and decision-making under resource limitations and find that such limitations induce discontinuous and non-monotonic phase transitions between memory-less and memory-based strategies. Therefore, this theory establishes a comprehensive framework for addressing biological information processing and decision-making under resource limitations, revealing the rich and complex behaviors that arise from resource limitations.

Figures (15)

• Figure 1: Schematic diagram of diverse information processing and resource limitations. Resource limitations become more pronounced in primitive organisms such as bacteria. A part of this figure is from TogoTV ($\copyright$ 2016 DBCLS TogoTV, CC-BY-4.0https://creativecommons.org/licenses/by/4.0/).
• Figure 2: Schematic diagrams of (a) the conventional optimal estimation theory without resource limitations and (b) the proposed optimal estimation theory with resource limitations. (a) In the conventional Bayesian filtering theory, an agent compresses the information about past observation history $y_{0:t-dt}:=\{y_{0},...,y_{t-dt}\}$ into posterior probability $p(x_{t}|y_{0:t-dt})$ and estimates environmental state $x_{t}$ from current observation $y_{t}$ and posterior probability $p(x_{t}|y_{0:t-dt})$. To obtain posterior probability $p(x_{t}|y_{0:t-dt})$, the agent needs to compute infinite-dimensional and deterministic Bayesian update. In addition, optimal state estimator function $\hat{x}$ is determined by minimizing only the state estimation error. These facts ignore the resource limitations of the agent. (b) In the optimal memory control theory, the agent compresses the information about past observation history $y_{0:t-dt}$ into finite-dimensional memory $z_{t}$ and estimates environmental state $x_{t}$ from current observation $y_{t}$ and memory $z_{t}$. The memory dynamics includes a noise process, $\xi_{t}$, which represents the intrinsic noise of the agent. Furthermore, the memory dynamics is optimized by optimal memory control $v_{t}^{*}$ rather than by Bayes' theorem. Optimal state estimator function $\hat{x}^{*}$ and optimal memory control function $v^{*}$ are determined by minimizing not only the state estimation error but also the memory control cost. This problem formulation allows us to take into account the resource limitations of the agent.
• Figure 3: Schematic diagram of a minimal model of biological information processing. (a) A cell achieves adaptive behavior by accurately perceiving environmental ligands through receptor activities and intracellular chemical reactions. The environmental ligands, receptor activities, and intracellular chemical reactions correspond to the state $x_{t}\in\mathbb{R}$, observation $y_{t}\in\mathbb{R}$, and memory $z_{t}\in\mathbb{R}$, respectively. (b-f) Stochastic simulation of state $x_{t}$ (b), observation $y_{t}$ (c), memory $z_{t}$ (d), optimal state estimator $\hat{x}_{t}^{*}$ (e), and optimal memory control $v_{t}^{*}$ (f). Optimal state estimator $\hat{x}_{t}^{*}$ and optimal memory control $v_{t}^{*}$ are determined based on observation $y_{t}$ and memory $z_{t}$ (orange and blue arrows, respectively). Optimal memory control $v_{t}^{*}$ shapes the optimal dynamics of memory $z_{t}$ (red arrow). For comparison, true state $x_{t}$ (black line) and estimated state $\hat{x}_{t}^{*}$ (orange line) are plotted on the same panel (e). The parameters are $D=100$, $E=500$, $F=1$, $Q=10$, and $M=1$.
• Figure 4: Phase transition in optimal estimation strategy. (a,b) Schematic diagrams of estimation strategies without memory (a) and with memory (b). (a) When $\Pi_{xz}=\Pi_{zz}=0$, the past observation information is not encoded into memory $z_{t}$, and state $x_{t}$ is estimated only from observation $y_{t}$. (b) When $\Pi_{xz},\Pi_{zz}\neq0$, the past observation information is encoded into memory $z_{t}$, and state $x_{t}$ is estimated from observation $y_{t}$ and memory $z_{t}$. (c) Expressions of optimal memory control $v_{t}^{*}$ and objective function $J$. (d--i) Memory control gains $\Pi_{zx}$ and $\Pi_{zz}$ and associated objective function $J$ with respect to observation noise $E$ (d--f) and memory noise $F$ (g--i). The blue, brown, and orange dots are the solutions of Pontryagin's minimum principle, corresponding to zero, intermediate, and high memory controls, respectively. The blue dots are optimal in the blue region, while the orange dots are optimal in the orange region. There are no regions where the brown dots are optimal. The parameters in (d--f) are $D=100$, $F=1$, $Q=10$, and $M=1$, while those in (g--i) are $D=100$, $E=10$, $Q=10$, and $M=1$.
• Figure 5: Objective function $J$ (a,d,g), state estimation error $J_{Q}$ (b,e,h), and memory control cost $J_{M}$ (c,f,i) with respect to memory control gains $\Pi_{zx}$ and $\Pi_{zz}$ under different observation noise $E$ and memory noise $F$. The blue, brown, and orange stars are the solutions of Pontryagin's minimum principle. The white dashed curves represent $J_{M}=3$, which clarify the changes in $J_{M}$. The rest of the parameters are $D=100$, $Q=10$, and $M=1$.