Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Tucker Tensor Train Taylor Series

Nick Alger, Blake Christierson, Peng Chen, Omar Ghattas

Abstract

We present methods for constructing Taylor series surrogate models for covariance preconditioned high dimensional mappings that depend implicitly on the solution of a system of nonlinear equations, e.g., the solution of a partial differential equation. Taylor series are traditionally considered intractable for such mappings because the derivative tensors are enormous, and are only accessible through ``probing'' (contraction of the tensor with vectors in all but one index). We overcome these challenges using a ``Tucker tensor train Taylor series'' (T4S) surrogate model, in which each derivative tensor is approximated by a Tucker decomposition composed with a tensor train. After an initial dimension reduction, Tucker tensor trains are fit to directionally symmetric tensor probes using Riemannian manifold optimization within a rank continuation scheme. The optimization is enabled by fast sweeping methods for applying the Riemannian Jacobian (the Jacobian for the Tucker tensor train fitting problem) and its transpose to vectors. We justify the T4S model theoretically, and provide numerical evidence for the effectiveness of the proposed methods.

Tucker Tensor Train Taylor Series

Abstract

We present methods for constructing Taylor series surrogate models for covariance preconditioned high dimensional mappings that depend implicitly on the solution of a system of nonlinear equations, e.g., the solution of a partial differential equation. Taylor series are traditionally considered intractable for such mappings because the derivative tensors are enormous, and are only accessible through ``probing'' (contraction of the tensor with vectors in all but one index). We overcome these challenges using a ``Tucker tensor train Taylor series'' (T4S) surrogate model, in which each derivative tensor is approximated by a Tucker decomposition composed with a tensor train. After an initial dimension reduction, Tucker tensor trains are fit to directionally symmetric tensor probes using Riemannian manifold optimization within a rank continuation scheme. The optimization is enabled by fast sweeping methods for applying the Riemannian Jacobian (the Jacobian for the Tucker tensor train fitting problem) and its transpose to vectors. We justify the T4S model theoretically, and provide numerical evidence for the effectiveness of the proposed methods.
Paper Structure (78 sections, 10 theorems, 232 equations, 24 figures, 1 table, 4 algorithms)

This paper contains 78 sections, 10 theorems, 232 equations, 24 figures, 1 table, 4 algorithms.

Table of Contents

  1. Introduction
  2. T4S model
  3. Tensor probes
  4. Overview of the paper
  5. Tensor background
  6. Tensor correspondences
  7. Scalar and vector valued multilinear functions
  8. Arrays and multilinear functions
  9. Linear algebra and norms for tensors
  10. Matrix unfoldings and matricizations
  11. Tucker tensor trains
  12. Derivative probing
  13. Derivative notation
  14. Forward probing notation
  15. Reverse probing notation
  16. ...and 63 more sections

Key Result

Lemma 1

Suppose ${\bm{A}}$ is an $N_1 \times \dots \times N_d$ array with corresponding scalar valued multilinear function $A$, ${\bm{M}}_i$ are $N_i \times q_i$ matrices, and $F$ is the scalar valued multilinear function with corresponding $q_1 \times \dots \times q_d$ array ${\bm{F}}$. Further, let ${\bm{F}}_\text{mat}$ and ${\bm{A}}_\text{mat}$ denote the $i$th matrix unfoldings of ${\bm{F}}$ and ${\b

Figures (24)

  • Figure 1:
  • Figure 2:
  • Figure 3:
  • Figure 4: Matrix unfoldings (top) and matricizations (bottom) of an $N_1 \times N_2 \times N_3 \times N_4$ array. The zeroth unfolding (row vector) and $4$th unfolding (column vector) are not shown.
  • Figure 5: Top: Cores for the tensor train ${\mathbb{S}}=({\bm{G}}_i)_{i=1}^{5}$. Bottom: Contracted tensor train in graphical tensor notation.
  • ...and 19 more figures

Theorems & Definitions (24)

  • Definition 1: T4S model
  • Definition 2: Tensor probes
  • Lemma 1: Matrix unfoldings and the Kronecker product
  • Definition 3: Tensor train (TT)
  • Definition 4: Tucker tensor train (T3)
  • Proposition 2: Preconditioned multilinear maps are tensor trains
  • proof
  • Lemma 3: Helper for \ref{['thm:preconditioned_are_tt']}: peeling off the last two modes of $T$
  • proof
  • Lemma 4: Helper for \ref{['thm:preconditioned_are_tt']}: first step of the peeling process
  • ...and 14 more