Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Tensor Neural Network Interpolation and Its Applications

Yongxin Li, Zhongshuo Lin, Yifan Wang, Hehu Xie

TL;DR

This work tackles the curse of dimensionality in high-dimensional PDEs and integrations by introducing tensor neural networks (TNNs) that realize a low-rank, tensor-product structure for otherwise non-tensor-product-type functions. By developing a TNN-based interpolation method, the authors transform non-tensor-product coefficients and sources into tensor-product representations, enabling efficient high-dimensional quadrature and the use of Ritz-type losses in PDE solvers. They prove a universal approximation property in $H^m$ and provide an error bound for elliptic problems, then demonstrate with two numerical experiments that TNN interpolation delivers high-accuracy high-dimensional integrations (e.g., $d=8$) and PDE solutions with non-tensor-product data (up to $d=20$). The approach offers a principled, scalable pathway to accurate, data-driven handling of complex high-dimensional problems with potential broad impact in physics, engineering, and finance.

Abstract

Based on tensor neural network, we propose an interpolation method for high dimensional non-tensor-product-type functions. This interpolation scheme is designed by using the tensor neural network based machine learning method. This means that we use a tensor neural network to approximate high dimensional functions which has no tensor product structure. In some sense, the non-tenor-product-type high dimensional function is transformed to the tensor neural network which has tensor product structure. It is well known that the tensor product structure can bring the possibility to design highly accurate and efficient numerical methods for dealing with high dimensional functions. In this paper, we will concentrate on computing the high dimensional integrations and solving high dimensional partial differential equations. The corresponding numerical methods and numerical examples will be provided to validate the proposed tensor neural network interpolation.

Tensor Neural Network Interpolation and Its Applications

TL;DR

This work tackles the curse of dimensionality in high-dimensional PDEs and integrations by introducing tensor neural networks (TNNs) that realize a low-rank, tensor-product structure for otherwise non-tensor-product-type functions. By developing a TNN-based interpolation method, the authors transform non-tensor-product coefficients and sources into tensor-product representations, enabling efficient high-dimensional quadrature and the use of Ritz-type losses in PDE solvers. They prove a universal approximation property in and provide an error bound for elliptic problems, then demonstrate with two numerical experiments that TNN interpolation delivers high-accuracy high-dimensional integrations (e.g., ) and PDE solutions with non-tensor-product data (up to ). The approach offers a principled, scalable pathway to accurate, data-driven handling of complex high-dimensional problems with potential broad impact in physics, engineering, and finance.

Abstract

Based on tensor neural network, we propose an interpolation method for high dimensional non-tensor-product-type functions. This interpolation scheme is designed by using the tensor neural network based machine learning method. This means that we use a tensor neural network to approximate high dimensional functions which has no tensor product structure. In some sense, the non-tenor-product-type high dimensional function is transformed to the tensor neural network which has tensor product structure. It is well known that the tensor product structure can bring the possibility to design highly accurate and efficient numerical methods for dealing with high dimensional functions. In this paper, we will concentrate on computing the high dimensional integrations and solving high dimensional partial differential equations. The corresponding numerical methods and numerical examples will be provided to validate the proposed tensor neural network interpolation.
Paper Structure (10 sections, 2 theorems, 32 equations, 2 figures, 2 tables, 1 algorithm)

This paper contains 10 sections, 2 theorems, 32 equations, 2 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Integration accuracy and machine learning accuracy
  3. Tensor neural network architecture
  4. Tensor neural network interpolation
  5. TNN interpolation for high dimensional integration
  6. TNN interpolation for solving high dimensional PDEs
  7. Numerical examples
  8. High dimensional integration
  9. TNN interpolation for solving partial differential equation
  10. Conclusion

Key Result

Theorem 3.1

Assume that each $\Omega_i$ is an interval in $\mathbb R$ for $i=1, \cdots, d$, $\Omega=\Omega_1\times\cdots\times\Omega_d$, and the function $f(x)\in H^m(\Omega)$. Then for any tolerance $\varepsilon>0$, there exist a positive integer $p$ and the corresponding TNN defined by (def_TNN) such that the

Figures (2)

  • Figure 1: Relative errors of four methods during the training process.
  • Figure 2: Architecture of TNN. Black arrows mean linear transformation (or affine transformation). Each ending node of blue arrows is obtained by taking the scalar multiplication of all starting nodes of blue arrows that end in this ending node. The final output of TNN is derived from the summation of all starting nodes of red arrows.

Theorems & Definitions (3)

  • Theorem 3.1
  • Theorem 6.1
  • proof