Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Information-theoretic multi-time-scale partially observable systems with inspiration from leukemia treatment

Margaret P. Chapman, Emily Jensen, Steven M. Chan, Laurent Lessard

TL;DR

The paper addresses control of a discrete-time partially observable nonlinear stochastic system with unknown parameters $\theta$, where state and measurement time scales differ, motivated by leukemia treatment. It develops a unifying theoretical framework that blends POMDPs, optimum experiment design via the Fisher information matrix, and stochastic dual/adaptive control, using an augmented belief state $\chi_t=(x_t^\top,\xi_t^\top,\hat{\theta}_t^\top)^\top$. Information-theoretic costs enter the objective as $c_t(\chi_t,u_t)=\hat{c}_t(x_t,u_t;\hat{\theta}_t)+\lambda \bar{c}_t(\chi_t)$ with $\bar{c}_t(\chi_t)=-\min\{\operatorname{trace}(\mathcal{F}_t(\chi_t)), b\}$, promoting informative measurements and parameter estimation. Under regularity assumptions (A1), there exists an optimal policy in the belief-space MDP, and the conceptual leukemia-treatment model demonstrates the framework's potential for data-driven, multi-time-scale dosing strategies and bridging theory to practice in biomedical control.

Abstract

We study a partially observable nonlinear stochastic system with unknown parameters, where the given time scales of the states and measurements may be distinct. The proposed setting is inspired by disease management, particularly leukemia.

Information-theoretic multi-time-scale partially observable systems with inspiration from leukemia treatment

TL;DR

The paper addresses control of a discrete-time partially observable nonlinear stochastic system with unknown parameters , where state and measurement time scales differ, motivated by leukemia treatment. It develops a unifying theoretical framework that blends POMDPs, optimum experiment design via the Fisher information matrix, and stochastic dual/adaptive control, using an augmented belief state . Information-theoretic costs enter the objective as with , promoting informative measurements and parameter estimation. Under regularity assumptions (A1), there exists an optimal policy in the belief-space MDP, and the conceptual leukemia-treatment model demonstrates the framework's potential for data-driven, multi-time-scale dosing strategies and bridging theory to practice in biomedical control.

Abstract

We study a partially observable nonlinear stochastic system with unknown parameters, where the given time scales of the states and measurements may be distinct. The proposed setting is inspired by disease management, particularly leukemia.
Paper Structure (12 sections, 3 theorems, 40 equations, 2 figures, 1 table)

This paper contains 12 sections, 3 theorems, 40 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related literature
  3. Notation
  4. Preliminaries
  5. System description
  6. Information-theoretic control system
  7. Formulating information-theoretic costs
  8. Defining $\chi_t$-dynamics
  9. Representation as a regular POMDP
  10. Policy Synthesis
  11. Conceptual example
  12. Conclusion

Key Result

Theorem 1

Let A1 hold. Then, $c_t$finalcosts is l.s.c. and b.b. for every $t \in \mathbb{T}_N$, and $F_t$mynewFta is continuous for every $t \in \mathbb{T}$. In particular, $G_t$myGt is continuous for every $t \in \mathbb{T} \setminus \mathbb{T}_y$.

Figures (2)

  • Figure 1: A high-level illustration of this work.
  • Figure 2: An illustration of the time scales in the conceptual leukemia treatment example. The close-up illustrates $\mathbb{T}$, where the time steps are hourly. For the illustration, we have simulated the mature white blood cells using a nominal model (to be presented in Eq. \ref{['fbarexample']}) and the dosing regimen from deangelo2015long. In the simulation, we have initialized the states to be the equilibrium values of the dynamics when the drug input is zero; parameter values are from jayachandran2014optimaljayachandran2015model.

Theorems & Definitions (18)

  • Remark 1: Weak convergence
  • Remark 2: Stochastic kernels
  • Remark 3: Some continuity facts
  • Remark 4: Kernel $\delta_{f}$
  • Remark 5: Measurable selection
  • Theorem 1: Regularity of $c_t$, $F_t$, and $G_t$
  • Definition 1: Multi-time-scale regular POMDP
  • Definition 2: State transition kernels $q_1,\dots, q_N$
  • Definition 3: Observation kernel $s_t$ for $t\in \mathbb{T}_y$
  • Lemma 1: Regularity of $q_t$ and $s_t$
  • ...and 8 more