Adaptive planning for risk-aware predictive digital twins

TL;DR

The paper tackles robustness of predictive digital twins under rare events by embedding a probabilistic graphical model within a parametric Markov decision process and deploying risk measures such as CVaR. It employs a dynamic Bayesian network with Bayesian updates to transition probabilities $Q\sim\mathcal{B}e(\alpha,\beta)$, and uses model checking (e.g., Storm) to synthesize policies that satisfy safety constraints while minimizing expected costs. Through UAV-inspired case studies, it demonstrates online policy refinement using CVaR and MAP estimates, achieving substantial cost reductions (about $22\%$) and improved digital state predictions. The approach advances reliable predictive maintenance and adaptive replanning by coupling uncertainty quantification, risk-aware planning, and real-time data assimilation in digital twins.

Abstract

This work proposes a mathematical framework to increase the robustness to rare events of digital twins modelled with graphical models. We incorporate probabilistic model-checking and linear programming into a dynamic Bayesian network to enable the construction of risk-averse digital twins. By modeling with a random variable the probability of the asset to transition from one state to another, we define a parametric Markov decision process. By solving this Markov decision process, we compute a policy that defines state-dependent optimal actions to take. To account for rare events connected to failures we leverage risk measures associated with the distribution of the random variables describing the transition probabilities. We refine the optimal policy at every time step resulting in a better trade off between operational costs and performances. We showcase the capabilities of the proposed framework with a structural digital twin of an unmanned aerial vehicle and its adaptive mission replanning.

Figures (12)

• Figure 1: Abstract representation of the information flow within the DT formulation, from sensor data acquisition to the MDP's optimal policy update. Starting from strain sensors placed on the wing, we estimate the digital state connected with the structural integrity of the UAV. We then unroll the graphical model with all the variables for a new time step and we update the posterior estimates of the transition probability. We recompute the new optimal policy and we issue a new action to perform. This policy is also used to compute future actions in the prediction phase happening in the digital space.
• Figure 2: Representation of the dynamic decision network used to encode the relationships between the physical and the digital spaces for the first 3 time steps. Square nodes denote actions, while diamond nodes denote the reward function. Bold outlines represent observed (deterministic) quantities, while thin outlines stand for estimated (random) variables.
• Figure 3: Dynamic Bayesian network used for the prediction of the digital state evolution with the associated uncertainty. We assume to have the estimation of $D_{t_c}$ given the observations from the physical asset $O_{t_c}$, the previous issued action $U_{t_c-1}$, and the previous digital state $D_{t_c-1}$. From the current time $t_c$ we show the graph used to predict the next 3 actions and digital states, using the policy $\pi_{t_c}$.
• Figure 4: Illustration of the types of transition matrices considered in this work. On the left panel the matrix for the actions in $\mathcal{A}^D$, where green stands for $1$ and white for $0$. In the central panel we have the upper bidiagonal with fixed transitions for actions in $\mathcal{A}^N$, where red stands for $1-q$ and blue for $q$. On the right panel the generic upper bidiagonal for transition probabilities varying between states.
• Figure 5: Maximum a posteriori (MAP) estimate, value at risk (VaR), and conditional value at risk (CVaR) for a generic probability distribution followed by the random variable $Q$.