Predictive Monitoring of Black-Box Dynamical Systems

TL;DR

TPM addresses predictive runtime monitoring of black-box dynamical systems with quantitative safety properties. It introduces Taylor-based Predictive Monitoring (TPM) that learns a local polynomial model from past observations using Taylor expansion and backward-difference derivatives to forecast future states and safety levels over a horizon $h$ with sampling interval $\tau$. The authors provide formal error bounds under the assumption that the system dynamics and controller are $(l+1)$-times differentiable and show that TPM is computationally lightweight and more accurate than the time-to-collision baseline in experiments on F1Tenth and F-16. The results support proactive safety verification with potential for extensions to higher-order differences, stochastic dynamics, and multi-agent settings.

Abstract

We study the problem of predictive runtime monitoring of black-box dynamical systems with quantitative safety properties. The black-box setting stipulates that the exact semantics of the dynamical system and the controller are unknown, and that we are only able to observe the state of the controlled (aka, closed-loop) system at finitely many time points. We present a novel framework for predicting future states of the system based on the states observed in the past. The numbers of past states and of predicted future states are parameters provided by the user. Our method is based on a combination of Taylor's expansion and the backward difference operator for numerical differentiation. We also derive an upper bound on the prediction error under the assumption that the system dynamics and the controller are smooth. The predicted states are then used to predict safety violations ahead in time. Our experiments demonstrate practical applicability of our method for complex black-box systems, showing that it is computationally lightweight and yet significantly more accurate than the state-of-the-art predictive safety monitoring techniques.

Figures (4)

• Figure 1: Predictive runtime monitoring using our TPM (solid) and the baseline TTC (dashed). The solid blue line is the ground truth trajectory. Green and red represent predictions of, respectively, safe and unsafe behaviors within the horizon. As can be seen, TMP is more accurate in predicting smooth turns.
• Figure 2: Taylor-Based Predictive Monitor (TPM)
• Figure 3: Ablation test results for prediction errors on F1Tenth (top row) and F-16 (bottom row). The lines represent the mean whereas the shaded regions represent the spread. For TPM, the constant $l$ is the degree of the Taylor's polynomial. The lookahead (X-axes) are measured as $h\tau$ for varying $h$.
• Figure 4: Visualization of the monitor's output as compared to the ground truth trajectories for the first two state dimensions of F1Tenth with $l=2$ (top row) and four different state dimensions of F-16 with $l=3$ (middle and bottom rows).

Theorems & Definitions (6)

• theorem 1: Taylor's theorem rudin1964principles
• lemma 1
• proof : Proof of Lem. \ref{['lem:backward difference error']}
• remark 1
• theorem 2
• proof