Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Functional Tensor Decompositions for Physics-Informed Neural Networks

Sai Karthikeya Vemuri, Tim Büchner, Julia Niebling, Joachim Denzler

TL;DR

This work tackles the challenge of solving high-dimensional PDEs with Physics-Informed Neural Networks (PINNs) by introducing functional tensor decompositions to separate variables. By representing the multivariate solution as an outer product of univariate neural networks, the authors develop CP-PINN, TT-PINN, and Tucker-PINN architectures that learn per-dimension components and combine them according to canonical, TT, or Tucker decompositions. The method achieves significant accuracy gains and reduced collocation requirements on benchmarks such as the Klein-Gordon, Helmholtz, and 5D Poisson equations, demonstrating up to an order-of-magnitude improvement in efficiency over state-of-the-art PINNs. This tensor-decomposition-enhanced PINN framework mitigates the curse of dimensionality and extends the applicability of PINNs to higher-dimensional PDE problems with scalable performance.

Abstract

Physics-Informed Neural Networks (PINNs) have shown continuous and increasing promise in approximating partial differential equations (PDEs), although they remain constrained by the curse of dimensionality. In this paper, we propose a generalized PINN version of the classical variable separable method. To do this, we first show that, using the universal approximation theorem, a multivariate function can be approximated by the outer product of neural networks, whose inputs are separated variables. We leverage tensor decomposition forms to separate the variables in a PINN setting. By employing Canonic Polyadic (CP), Tensor-Train (TT), and Tucker decomposition forms within the PINN framework, we create robust architectures for learning multivariate functions from separate neural networks connected by outer products. Our methodology significantly enhances the performance of PINNs, as evidenced by improved results on complex high-dimensional PDEs, including the 3d Helmholtz and 5d Poisson equations, among others. This research underscores the potential of tensor decomposition-based variably separated PINNs to surpass the state-of-the-art, offering a compelling solution to the dimensionality challenge in PDE approximation.

Functional Tensor Decompositions for Physics-Informed Neural Networks

TL;DR

This work tackles the challenge of solving high-dimensional PDEs with Physics-Informed Neural Networks (PINNs) by introducing functional tensor decompositions to separate variables. By representing the multivariate solution as an outer product of univariate neural networks, the authors develop CP-PINN, TT-PINN, and Tucker-PINN architectures that learn per-dimension components and combine them according to canonical, TT, or Tucker decompositions. The method achieves significant accuracy gains and reduced collocation requirements on benchmarks such as the Klein-Gordon, Helmholtz, and 5D Poisson equations, demonstrating up to an order-of-magnitude improvement in efficiency over state-of-the-art PINNs. This tensor-decomposition-enhanced PINN framework mitigates the curse of dimensionality and extends the applicability of PINNs to higher-dimensional PDE problems with scalable performance.

Abstract

Physics-Informed Neural Networks (PINNs) have shown continuous and increasing promise in approximating partial differential equations (PDEs), although they remain constrained by the curse of dimensionality. In this paper, we propose a generalized PINN version of the classical variable separable method. To do this, we first show that, using the universal approximation theorem, a multivariate function can be approximated by the outer product of neural networks, whose inputs are separated variables. We leverage tensor decomposition forms to separate the variables in a PINN setting. By employing Canonic Polyadic (CP), Tensor-Train (TT), and Tucker decomposition forms within the PINN framework, we create robust architectures for learning multivariate functions from separate neural networks connected by outer products. Our methodology significantly enhances the performance of PINNs, as evidenced by improved results on complex high-dimensional PDEs, including the 3d Helmholtz and 5d Poisson equations, among others. This research underscores the potential of tensor decomposition-based variably separated PINNs to surpass the state-of-the-art, offering a compelling solution to the dimensionality challenge in PDE approximation.
Paper Structure (11 sections, 18 equations, 2 figures, 3 tables)

This paper contains 11 sections, 18 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Theoretical Background
  3. Universal Approximation Theorem
  4. Physics-Informed Neural Networks
  5. Functional Tensor Decompositions for PINNs
  6. Canonic-Polyadic Decomposition
  7. Tensor-Train Decomposition
  8. Tucker Decomposition
  9. Functional tensor decomposition forms in PINNs
  10. Experiments
  11. Discussion and Conclusions

Figures (2)

  • Figure 1: We provide a schematic visualization for the tensor decompositions (\ref{['fig:decomp_cp']})-(\ref{['fig:decomp_tu']}) on the examples for $d=3$. The shape of the factor tensors ($A$) is written on the bottom of each component. Tucker Tucker1966 additionally has one core tensor $C$.
  • Figure 2: Functional tensor decomposition forms within the PINN model architecture: The approximation of each component matrix based on a single variable is done with an individual neural network. These outputs are then combined as in the Canonic-Polyadic cphitchcock (\ref{['fig:cppinn']}), Tensor-Train ttdecomp (\ref{['fig:ttpinn']}) or Tucker Tucker1966 (\ref{['fig:tupinn']}) manner.