Digital Twins: McKean-Pontryagin Control for Partially Observed Physical Twins

TL;DR

This paper addresses online optimal control for partially observed diffusion processes within the digital twin paradigm by fusing data assimilation with the McKean–Pontryagin mean-field control framework. It replaces fully observed HJB/Pontryagin schemes with a forward-evolving system of interacting particles for states and co-states, whose dynamics depend on the current belief distribution $\pi_t$ derived from an ensemble Kalman filter. The authors derive continuous- and discrete-time formulations, implement a Schrödinger-bridge–based interacting-particle algorithm, and demonstrate real-time control on a controlled Lorenz-63 system and an inverted pendulum, showing robustness to observation noise and finite ensemble sizes. The approach enables online adaptation of the digital twin and simultaneous computation of control laws, with potential impact on real-time digital twins in engineering and beyond.

Abstract

Optimal control for fully observed diffusion processes is well established and has led to numerous numerical implementations based on, for example, Bellman's principle, model free reinforcement learning, Pontryagin's maximum principle, and model predictive control. On the contrary, much fewer algorithms are available for optimal control of partially observed processes. However, this scenario is central to the digital twin paradigm where a physical twin is partially observed and control laws are derived based on a digital twin. In this paper, we contribute to this challenge by combining data assimilation in the form of the ensemble Kalman filter with the recently proposed McKean-Pontryagin approach to stochastic optimal control. We derive forward evolving mean-field evolution equations for states and co-states which simultaneously allow for an online assimilation of data as well as an online computation of control laws. The proposed methodology is therefore perfectly suited for real time applications of digital twins. We present numerical results for a controlled Lorenz-63 system and an inverted pendulum.

Figures (7)

• Figure 1: Controlled Lorenz-63 model. Displayed is the three-dimensional trajectory of the mean $\{m^x_t\}$ over the time interval $t \in [0,100]$. Left panel: control restricted to range $|U_t|\le 50$; right panel: control restricted to range $|U_t|\le 100$. After an initial transient, the trajectory enters a quasi-period orbit for the larger threshold value. The smaller threshold value still allows for some transitions to negative ${\rm x}_t$ values and the chaotic nature of the Lorenz-63 system is retained.
• Figure 2: Controlled Lorenz-63 model. First component of the mean $m_t^x \in \mathbb{R}^3$ and associated control term $U_t$ as a function of time. Left panels: control restricted to range $|U_t|\le 50$; right panels: control restricted to range $|U_t|\le 100$.
• Figure 3: Controlled Lorenz-63 model. Estimation error; i.e. root mean square error between physical twin states $X_t^\dagger$ and digital twin mean states $m_t^x$, as a function of time. Left panel: control restricted to range $|U_t|\le 50$; right panel: control restricted to range $|U_t|\le 100$.
• Figure 4: Controlled Lorenz-63 model. Same as Figure \ref{['fig1']} but for ensemble size $M=4$.
• Figure 5: Controlled Lorenz-63 model. Same as Figure \ref{['fig2']} but for ensemble size $M=4$.

Theorems & Definitions (1)

• Remark 3.1