Control of Medical Digital Twins with Artificial Neural Networks

TL;DR

The effectiveness of the proposed neural-network control method is illustrated and benchmarked against other methods with two widely-used agent-based model types, and the relevance of the method introduced here extends beyond medical digital twins to other complex dynamical systems.

Abstract

The objective of personalized medicine is to tailor interventions to an individual patient's unique characteristics. A key technology for this purpose involves medical digital twins, computational models of human biology that can be personalized and dynamically updated to incorporate patient-specific data collected over time. Certain aspects of human biology, such as the immune system, are not easily captured with physics-based models, such as differential equations. Instead, they are often multi-scale, stochastic, and hybrid. This poses a challenge to existing model-based control and optimization approaches that cannot be readily applied to such models. Recent advances in automatic differentiation and neural-network control methods hold promise in addressing complex control problems. However, the application of these approaches to biomedical systems is still in its early stages. This work introduces dynamics-informed neural-network controllers as an alternative approach to control of medical digital twins. As a first use case for this method, the focus is on agent-based models, a versatile and increasingly common modeling platform in biomedicine. The effectiveness of the proposed neural-network control method is illustrated and benchmarked against other methods with two widely-used agent-based model types. The relevance of the method introduced here extends beyond medical digital twins to other complex dynamical systems.

Figures (5)

• Figure 1: Predator-prey ABM dynamics. A snapshot of a three-species predator-prey ABM simulation with $51\times51$ grid cells. Green and light brown grid cells represent nutrient-rich and nutrient-poor regions, respectively.
• Figure 2: Control of predator-prey dynamics with an ANN. (a) To control a predator-prey ABM, we need to define suitable inputs and outputs for the ANN. Potential inputs are the population sizes $a_k$, $b_k$, and $c_k$ of species $A$, $B$, and $C$ at time $k$. We aim at directly managing the numbers of predator and prey, so there are two outputs $u_1$ and $u_2$. Using a problem-tailored straight-through estimator, the ANN outputs nonnegative integer-valued controls $u_1,u_2$ after subtracting the fractional part $\{[\cdot]^+\}$ of the positive part of the hidden-layer outputs. We use $\sigma$ and $\{x^+\}$ to indicate hidden-layer activations and the straight-through estimator, respectively. (b) The evolution of nutrient-rich lattice sites, prey, and predators. The vertical dashed grey line indicates the time at which the ANN controller is switched on. The controller aims at increasing the mean number of prey by 10% and reducing the mean number of predators by 50%. We used a $255\times 255$ grid and set $b_0=2500$, $c_0=1250$, $\alpha_1=4.0$, $\alpha_2=5.0$, $\lambda_1=4.0$, $\lambda_2=20.0$, and $\tau=30$wilensky1997netlogo. The initial proportion of nutrient-rich lattice sites is 50%. (c) The control outputs $u_1(b_k;\theta_1)$ (i.e., prey control) and $u_2(c_k;\theta_2)$ (i.e., predator control) as a function of the control time. (d) The values of $\theta_1$ and $\theta_2$ learned by different control methods (blue disk: ANN controller; blue triangle: mechanistic approach; blue inverted triangle: S-system approach). The black cross indicates the optimal values of the parameters $\theta_1,\theta_2$ found via a grid search and the orange dots indicate corresponding 1$\sigma$-intervals.
• Figure 3: Control of transient predator-prey dynamics with an ANN. (a) The evolution of nutrient-rich lattice sites, prey, and predators. The vertical dashed grey line indicates the time at which the ANN controller is switched on. The controller aims at increasing the mean number of prey by 10% and reducing the mean number of predators by 50%. We used a $255\times 255$ grid and set $b_0=2500$, $c_0=1250$, $\alpha_1=4.0$, $\alpha_2=5.0$, $\lambda_1=4.0$, $\lambda_2=20.0$, and $\tau=30$. The initial proportion of nutrient-rich lattice sites is 50%. (b) The control outputs $u_1(b_k;\boldsymbol{\theta})$ (i.e., prey control) and $u_2(c_k;\boldsymbol{\theta})$ (i.e., predator control) as a function of the control time.
• Figure 4: Learning and controlling metabolic pathway dynamics with neural ODEs. (a) Overview of the reactions in the metabolic pathway model. There are four reactions associated with five metabolites ($S$, $P$, $Q$, $R$, and $T$) and four enzymes ($A$, $E$, $I$, and $O$). The two arrows originating from metabolite $R$ indicate that it inhibits enzyme $A$ and increases the rate of enzyme $O$. In our ABM simulations, all reactions are modelled at the microscale level. The initial amounts of metabolites $S$, $P$, $Q$, $R$, and $T$ are $8\times 10^4$, $2\times 10^4$, $2\times 10^4$, $10$, and $10$, respectively. The initial amount of each of the four enzymes is $200$. (b). The loss of the metabolic pathway control problem [see \ref{['eq:loss_metabolic']}] as a function of the inflow of substrate $S$ [blue dots: ABM; solid line: mechanistic metamodel; dashed line: generalized mass action (GMA) metamodel; dash-dotted line: neural ODE metamodel]. The blue-shaded regions indicate $1\sigma$-intervals that are based on 100 ABM instantiations. Minimizing the loss function means minimizing substrate depletion and maximizing the production of reaction products.
• Figure 5: Controlling metabolic pathway dynamics with a neural ODE controller. We show the evolution of the amount of substrate $S$ , the amounts of metabolites $R$ and $T$, and the control signal. The control signal is generated by a neural ODE controller, which we trained using a mechanistic Michaelis--Menten metamodel and the loss as defined in \ref{['eq:loss_metabolic']}. The shown amounts of $S$, $R$, and $T$ are averages over 100 ABM instantiations. The initial amounts of metabolites $S$, $R$, and $T$ are $8\times 10^4$, $10$, and $10$, respectively. In the shown plot, we rescaled these quantities by $10^{-4}$.