Learning About Structural Errors in Models of Complex Dynamical Systems
Jin-Long Wu, Matthew E. Levine, Tapio Schneider, Andrew Stuart
TL;DR
The paper addresses learning structural errors in semi-empirical closures for complex dynamical systems with unresolved scales. It generalizes external bias correction to an internal error term via $\dot{X} = f(X; \theta_P) + \delta(X; \theta_I)$, and develops derivative-free Kalman inversion and data-assimilation workflows for direct and indirect data. Contributions include a formal comparison of external vs internal error approaches, a suite of error-model constructions (dictionary, Gaussian processes, neural networks, stochastic and non-local extensions), and calibration methods with initial-condition handling and physics-based constraints, demonstrated on Lorenz-96 and a glucose-insulin model to improve long-time statistics and predictive accuracy. The framework provides a practical, data-informed pathway to enhance closures in systems without clear scale separation, potentially yielding more reliable forecasts and insights across physics, physiology, and engineering domains.
Abstract
Complex dynamical systems are notoriously difficult to model because some degrees of freedom (e.g., small scales) may be computationally unresolvable or are incompletely understood, yet they are dynamically important. For example, the small scales of cloud dynamics and droplet formation are crucial for controlling climate, yet are unresolvable in global climate models. Semi-empirical closure models for the effects of unresolved degrees of freedom often exist and encode important domain-specific knowledge. Building on such closure models and correcting them through learning the structural errors can be an effective way of fusing data with domain knowledge. Here we describe a general approach, principles, and algorithms for learning about structural errors. Key to our approach is to include structural error models inside the models of complex systems, for example, in closure models for unresolved scales. The structural errors then map, usually nonlinearly, to observable data. As a result, however, mismatches between model output and data are only indirectly informative about structural errors, due to a lack of labeled pairs of inputs and outputs of structural error models. Additionally, derivatives of the model may not exist or be readily available. We discuss how structural error models can be learned from indirect data with derivative-free Kalman inversion algorithms and variants, how sparsity constraints enforce a "do no harm" principle, and various ways of modeling structural errors. We also discuss the merits of using non-local and/or stochastic error models. In addition, we demonstrate how data assimilation techniques can assist the learning about structural errors in non-ergodic systems. The concepts and algorithms are illustrated in two numerical examples based on the Lorenz-96 system and a human glucose-insulin model.