Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning embeddings of non-linear PDEs: the Burgers' equation

Pedro Tarancón-Álvarez, Leonid Sarieddine, Pavlos Protopapas, Raul Jimenez

TL;DR

This work presents a method to construct solution embedding spaces of nonlinear partial differential equations using a multi-head setup, and extracts non-degenerate information from them using principal component analysis (PCA).

Abstract

Embeddings provide low-dimensional representations that organize complex function spaces and support generalization. They provide a geometric representation that supports efficient retrieval, comparison, and generalization. In this work we generalize the concept to Physics Informed Neural Networks. We present a method to construct solution embedding spaces of nonlinear partial differential equations using a multi-head setup, and extract non-degenerate information from them using principal component analysis (PCA). We test this method by applying it to viscous Burgers' equation, which is solved simultaneously for a family of initial conditions and values of the viscosity. A shared network body learns a latent embedding of the solution space, while linear heads map this embedding to individual realizations. By enforcing orthogonality constraints on the heads, we obtain a principal-component decomposition of the latent space that is robust to training degeneracies and admits a direct physical interpretation. The obtained components for Burgers' equation exhibit rapid saturation, indicating that a small number of latent modes captures the dominant features of the dynamics.

Learning embeddings of non-linear PDEs: the Burgers' equation

TL;DR

This work presents a method to construct solution embedding spaces of nonlinear partial differential equations using a multi-head setup, and extracts non-degenerate information from them using principal component analysis (PCA).

Abstract

Embeddings provide low-dimensional representations that organize complex function spaces and support generalization. They provide a geometric representation that supports efficient retrieval, comparison, and generalization. In this work we generalize the concept to Physics Informed Neural Networks. We present a method to construct solution embedding spaces of nonlinear partial differential equations using a multi-head setup, and extract non-degenerate information from them using principal component analysis (PCA). We test this method by applying it to viscous Burgers' equation, which is solved simultaneously for a family of initial conditions and values of the viscosity. A shared network body learns a latent embedding of the solution space, while linear heads map this embedding to individual realizations. By enforcing orthogonality constraints on the heads, we obtain a principal-component decomposition of the latent space that is robust to training degeneracies and admits a direct physical interpretation. The obtained components for Burgers' equation exhibit rapid saturation, indicating that a small number of latent modes captures the dominant features of the dynamics.
Paper Structure (16 sections, 4 equations, 2 figures)

This paper contains 16 sections, 4 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Related work (brief).
  3. Multihead PINNs as a learned basis
  4. Multihead decomposition with linear heads
  5. Multihead setup.
  6. Training objective.
  7. Head orthogonalization and identifiable PCA
  8. How we compute PCA.
  9. Experiments and results
  10. Implementation details.
  11. Fourier initial conditions
  12. Polynomial initial conditions
  13. Why embeddings matter for PDE--ML
  14. Discussion: multiscale organization and an effective theory
  15. Conclusion
  16. ...and 1 more sections

Figures (2)

  • Figure 1: Left: PINN PDE-residual loss versus epochs (Fourier IC ensemble). Right: Explained-variance ratio and cumulative variance of latent-space PCA (Fourier IC ensemble).
  • Figure 2: Left: PINN PDE-residual loss versus epochs (polynomial IC ensemble). Right: Explained-variance ratio and cumulative variance of latent-space PCA (polynomial IC ensemble).