Dual-Level Models for Physics-Informed Multi-Step Time Series Forecasting

TL;DR

The paper tackles the challenge of accurate multi-step forecasting for dynamical systems by integrating probabilistic input forecasting with physics-informed output prediction. It introduces a dual-level framework that first generates probabilistic input forecasts using Gaussian process state-space models, including hybrid LSTM-enhanced variants, and then propagates these inputs through discrete-time physics-informed neural networks to produce multi-step outputs with uncertainty quantification. The approach is instantiated on three dynamical systems (CSTR, ADPFR, and froth flotation) and shows that hybrid input forecasts yield higher log-likelihood and lower MSE, while PINNs driven by these inputs outperform purely data-driven models in both accuracy and generalization. Overall, the combination of hybrid input forecasting and PINN-based output prediction provides superior predictive distributions and point estimates, with significant potential for improved decision-making in process control and optimization.

Abstract

This paper develops an approach for multi-step forecasting of dynamical systems by integrating probabilistic input forecasting with physics-informed output prediction. Accurate multi-step forecasting of time series systems is important for the automatic control and optimization of physical processes, enabling more precise decision-making. While mechanistic-based and data-driven machine learning (ML) approaches have been employed for time series forecasting, they face significant limitations. Incomplete knowledge of process mathematical models limits mechanistic-based direct employment, while purely data-driven ML models struggle with dynamic environments, leading to poor generalization. To address these limitations, this paper proposes a dual-level strategy for physics-informed forecasting of dynamical systems. On the first level, input variables are forecast using a hybrid method that integrates a long short-term memory (LSTM) network into probabilistic state transition models (STMs). On the second level, these stochastically predicted inputs are sequentially fed into a physics-informed neural network (PINN) to generate multi-step output predictions. The experimental results of the paper demonstrate that the hybrid input forecasting models achieve a higher log-likelihood and lower mean squared errors (MSE) compared to conventional STMs. Furthermore, the PINNs driven by the input forecasting models outperform their purely data-driven counterparts in terms of MSE and log-likelihood, exhibiting stronger generalization and forecasting performance across multiple test cases.

Figures (10)

• Figure 1: Schematic of recursive multistep forecasting. The model $\hat{f}$ is applied iteratively, with predictions fed back as inputs for subsequent steps. The composition of the feature window evolves as the forecast horizon increases, transitioning from windows containing only observed target values (top) to windows containing predicted target values (bottom). Exogenous variables $\mathbf{x}_t$ appear at each time step, representing either known inputs or independent forecasts.
• Figure 2: Predictive performance of PINNs and their purely data-driven counterparts on test datasets across three dynamical systems: (a-b) CSTR concentration, (c-d) ADPFR concentration profiles at twelve randomly selected temporal snapshots, and (e-f) froth flotation concentrate grade. The left column shows PINN predictions, and the right column shows FFNN predictions. Across all systems, PINNs demonstrate superior accuracy in capturing system dynamics.
• Figure 3: Performance of state transition models in predicting CSTR inlet concentration ($C_{in}$) over a 10-step horizon. Conventional models are compared with hybrid LSTM-integrated counterparts. The shaded region shows the 95% confidence interval (CI).
• Figure 4: Comparison of state transition models for 10-step ahead forecasting of ADPFR superficial velocity ($v$). Mean predictions of conventional and LSTM‑integrated models are shown with the 95% confidence interval.
• Figure 5: Forecasting performance for flotation concentrate flow rate ($Q_c$) using state transition models over 10 steps. Both conventional and LSTM‑integrated approaches are shown with the 95% confidence interval.