Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Online learning of quadratic manifolds from streaming data for nonlinear dimensionality reduction and nonlinear model reduction

Paul Schwerdtner, Prakash Mohan, Aleksandra Pachalieva, Julie Bessac, Daniel O'Malley, Benjamin Peherstorfer

TL;DR

A range of numerical examples demonstrate that the proposed online greedy method learns accurate quadratic manifold embeddings while being capable of processing data that far exceed common disk input/output capabilities and volumes as well as main-memory sizes.

Abstract

This work introduces an online greedy method for constructing quadratic manifolds from streaming data, designed to enable in-situ analysis of numerical simulation data on the Petabyte scale. Unlike traditional batch methods, which require all data to be available upfront and take multiple passes over the data, the proposed online greedy method incrementally updates quadratic manifolds in one pass as data points are received, eliminating the need for expensive disk input/output operations as well as storing and loading data points once they have been processed. A range of numerical examples demonstrate that the online greedy method learns accurate quadratic manifold embeddings while being capable of processing data that far exceed common disk input/output capabilities and volumes as well as main-memory sizes.

Online learning of quadratic manifolds from streaming data for nonlinear dimensionality reduction and nonlinear model reduction

TL;DR

A range of numerical examples demonstrate that the proposed online greedy method learns accurate quadratic manifold embeddings while being capable of processing data that far exceed common disk input/output capabilities and volumes as well as main-memory sizes.

Abstract

This work introduces an online greedy method for constructing quadratic manifolds from streaming data, designed to enable in-situ analysis of numerical simulation data on the Petabyte scale. Unlike traditional batch methods, which require all data to be available upfront and take multiple passes over the data, the proposed online greedy method incrementally updates quadratic manifolds in one pass as data points are received, eliminating the need for expensive disk input/output operations as well as storing and loading data points once they have been processed. A range of numerical examples demonstrate that the online greedy method learns accurate quadratic manifold embeddings while being capable of processing data that far exceed common disk input/output capabilities and volumes as well as main-memory sizes.
Paper Structure (16 sections, 1 theorem, 30 equations, 10 figures, 1 algorithm)

This paper contains 16 sections, 1 theorem, 30 equations, 10 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Linear dimensionality reduction with encoder and decoder functions
  4. Dimensionality reduction with quadratic manifolds
  5. Greedy construction of quadratic manifolds from data
  6. Problem formulation
  7. Online learning of quadratic manifolds
  8. Greedy construction from the SVD of the data matrix
  9. Greedy construction from a truncated SVD of the data matrix
  10. Incremental updates to truncated SVD of the data matrix
  11. Online greedy construction of quadratic manifolds from streaming data
  12. Numerical experiments
  13. Hamiltonian interacting pulse
  14. Turbulent channel flow at a high Reynolds numbers
  15. Petabyte-scale data of Kelvin-Helmholtz instability
  16. ...and 1 more sections

Key Result

Proposition 3.1

The Frobenius norm of the difference between the minimizers of eq:OurMethod:LstsqUsingSVD and eq:OurMethod:LstsqUsingTruncSVD is bounded by $\alpha\|{\boldsymbol{\Sigma}^{\breve{\mathcal{I}}\setminus\hat{\mathcal{I}}}}\|_F$, where

Figures (10)

  • Figure 1: Hamiltonian wave: (a) The proposed online greedy method constructs a quadratic manifold from streaming data that achieves orders of magnitude low approximation errors \ref{['eq:NumExp:RelError']} on test data than linear dimensionality reduction with the SVD. (b) As the reduced dimension $n$ of the quadratic manifold is increased, the truncation dimension $q$ has to be increased too so that the online greedy method can utilize higher-order left-singular vectors of the data matrix.
  • Figure 2: Hamiltonian wave: The online greedy method constructs quadratic manifolds from streaming data that achieve visibly better approximations of unseen test data than linear dimensionality reduction. Approximations shown for reduced dimension $n=20$ and time $t=6$.
  • Figure 3: Hamiltonian wave: The plots show that the truncation dimension $q$ and the reduced dimension $n$ have to increase in tandem so that the online greedy method can leverage higher-order left-singular vector for larger $n$.
  • Figure 4: Hamiltonian wave: Because the leading singular values are approximated well by the incremental SVD, it is sufficient to truncate the SVD early in the online greedy method. Increasing the truncation dimension $q$ reduces the error of the approximated singular values.
  • Figure 5: Channel flow: As the truncation dimension $q$ is increased, the online greedy method constructs quadratic manifolds from streaming data that achieve about 40% lower test errors \ref{['eq:NumExp:RelError']} than linear dimensionality reduction with the SVD.
  • ...and 5 more figures

Theorems & Definitions (2)

  • Proposition 3.1
  • proof