Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Operator Learning for Smoothing and Forecasting

Edoardo Calvello, Elizabeth Carlson, Nikola Kovachki, Michael N. Manta, Andrew M. Stuart

Abstract

Machine learning has opened new frontiers in purely data-driven algorithms for data assimilation in, and for forecasting of, dynamical systems; the resulting methods are showing some promise. However, in contrast to model-driven algorithms, analysis of these data-driven methods is poorly developed. In this paper we address this issue, developing a theory to underpin data-driven methods to solve smoothing problems arising in data assimilation and forecasting problems. The theoretical framework relies on two key components: (i) establishing the existence of the mapping to be learned; (ii) the properties of the operator learning architecture used to approximate this mapping. By studying these two components in conjunction, we establish the first universal approximation theorem for purely data-driven algorithms for both smoothing and forecasting of dynamical systems. We work in the continuous time setting, hence deploying neural operator architectures. The theoretical results are illustrated with experiments studying the Lorenz `63, Lorenz `96 and Kuramoto-Sivashinsky dynamical systems.

Operator Learning for Smoothing and Forecasting

Abstract

Machine learning has opened new frontiers in purely data-driven algorithms for data assimilation in, and for forecasting of, dynamical systems; the resulting methods are showing some promise. However, in contrast to model-driven algorithms, analysis of these data-driven methods is poorly developed. In this paper we address this issue, developing a theory to underpin data-driven methods to solve smoothing problems arising in data assimilation and forecasting problems. The theoretical framework relies on two key components: (i) establishing the existence of the mapping to be learned; (ii) the properties of the operator learning architecture used to approximate this mapping. By studying these two components in conjunction, we establish the first universal approximation theorem for purely data-driven algorithms for both smoothing and forecasting of dynamical systems. We work in the continuous time setting, hence deploying neural operator architectures. The theoretical results are illustrated with experiments studying the Lorenz `63, Lorenz `96 and Kuramoto-Sivashinsky dynamical systems.
Paper Structure (23 sections, 8 theorems, 56 equations, 9 figures, 6 tables)

This paper contains 23 sections, 8 theorems, 56 equations, 9 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Context and Literature Review
  3. Contributions and Outline
  4. Notation
  5. Dynamical System Setup
  6. Observability
  7. Observability Setup
  8. Observability-Rank Condition
  9. Observability of Lorenz '63 Model
  10. Universal Approximation of Smoothing and Forecasting Maps
  11. Universal Approximation Theorem
  12. Smoothing
  13. Forecasting
  14. Data-Driven Approximation of Smoothing and Forecasting Maps
  15. Experimental Set-Up
  16. ...and 8 more sections

Key Result

Theorem 3.1

Let $D \subset \mathbb{R}^d$ be a bounded domain with Lipschitz boundary, and fix integers $s,s'\geq 0$. If $\Psi^\dagger\colon C^{s}(\overline{D};\,\mathbb{R}^{r}) \to C^{s'}(\overline{D};\,\mathbb{R}^{r'})$ is a continuous operator and $K \subset C^{s}(\overline{D};\,\mathbb{R}^{r})$ a compact set

Figures (9)

  • Figure 1: Median and worst-case relative $L^2$ error samples for smoothing experiment involving prediction of $(y,z)$ from $x$.
  • Figure 2: Median and worst-case relative $L^2$ error samples for smoothing experiment involving prediction of $(x,y)$ from $z$.
  • Figure 3: Median and worst-case relative $L^2$ error samples in forecasting (a). Distribution of trajectory predictions under composition of the learned forecasting map (b).
  • Figure 4: Median and worst-case performance on the Lorenz '96 system for smoothing.
  • Figure 5: Pointwise errors relative to the root mean square of the ground truth (computed in space) for the sample in the test set achieving the median relative $L^2$ error.
  • ...and 4 more figures

Theorems & Definitions (20)

  • Remark 2.1
  • Remark 2.2: Observability in Control Theory
  • Theorem 3.1
  • Proposition 3.2: Existence of map $W:p\mapsto (p(0), q(0))$
  • Proof 1
  • Proposition 3.3: Existence of map $\Psi_S^\dagger:p\mapsto q$
  • Proof 2
  • Theorem 3.4: Universal Approximation for Smoothing
  • Proof 3
  • Remark 3.5
  • ...and 10 more