Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning Low Rank Neural Representations of Hyperbolic Wave Dynamics from Data

Woojin Cho, Kookjin Lee, Noseong Park, Donsub Rim, Gerrit Welper

TL;DR

This work introduces Low Rank Neural Representations (LRNR) as a data-driven, hypernetwork-enabled surrogate framework for hyperbolic wave dynamics. By enforcing a compositional, low-rank structure on network weights and using a hypernetwork to emit time-dependent LRNR coefficients, the method yields compact representations with provable efficiency bounds for wave, advection, and nonlinear conservation-law solutions. The authors demonstrate interpretability through hypermodes, show that coefficient dynamics are smooth and amenable to extrapolation, and present FastLRNR to achieve real-time evaluation. Spatial extrapolation and causality-respecting behavior are observed in multiple 2D examples, highlighting LRNRs’ potential for PDE surrogates and PDE-constrained learning.

Abstract

We present a data-driven dimensionality reduction method that is well-suited for physics-based data representing hyperbolic wave propagation. The method utilizes a specialized neural network architecture called low rank neural representation (LRNR) inside a hypernetwork framework. The architecture is motivated by theoretical results that rigorously prove the existence of efficient representations for this wave class. We illustrate through archetypal examples that such an efficient low-dimensional representation of propagating waves can be learned directly from data through a combination of deep learning techniques. We observe that a low rank tensor representation arises naturally in the trained LRNRs, and that this reveals a new decomposition of wave propagation where each decomposed mode corresponds to interpretable physical features. Furthermore, we demonstrate that the LRNR architecture enables efficient inference via a compression scheme, which is a potentially important feature when deploying LRNRs in demanding performance regimes.

Learning Low Rank Neural Representations of Hyperbolic Wave Dynamics from Data

TL;DR

This work introduces Low Rank Neural Representations (LRNR) as a data-driven, hypernetwork-enabled surrogate framework for hyperbolic wave dynamics. By enforcing a compositional, low-rank structure on network weights and using a hypernetwork to emit time-dependent LRNR coefficients, the method yields compact representations with provable efficiency bounds for wave, advection, and nonlinear conservation-law solutions. The authors demonstrate interpretability through hypermodes, show that coefficient dynamics are smooth and amenable to extrapolation, and present FastLRNR to achieve real-time evaluation. Spatial extrapolation and causality-respecting behavior are observed in multiple 2D examples, highlighting LRNRs’ potential for PDE surrogates and PDE-constrained learning.

Abstract

We present a data-driven dimensionality reduction method that is well-suited for physics-based data representing hyperbolic wave propagation. The method utilizes a specialized neural network architecture called low rank neural representation (LRNR) inside a hypernetwork framework. The architecture is motivated by theoretical results that rigorously prove the existence of efficient representations for this wave class. We illustrate through archetypal examples that such an efficient low-dimensional representation of propagating waves can be learned directly from data through a combination of deep learning techniques. We observe that a low rank tensor representation arises naturally in the trained LRNRs, and that this reveals a new decomposition of wave propagation where each decomposed mode corresponds to interpretable physical features. Furthermore, we demonstrate that the LRNR architecture enables efficient inference via a compression scheme, which is a potentially important feature when deploying LRNRs in demanding performance regimes.

Paper Structure

This paper contains 38 sections, 3 theorems, 86 equations, 18 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Low Rank Neural Representation
  3. The LRNR architecture
  4. Implementation version
  5. Approximation results on LRNR coefficient counts
  6. The LRNR coefficient count
  7. The $d$-dimensional wave equation
  8. Motivation
  9. LRNR coefficient count for the $d$-dimensional wave equation
  10. The $d$-dimensional advection equation
  11. The 1$d$ scalar conservation laws
  12. Data-driven meta-learning framework for wave problems
  13. Sparsity promoting regularizer and orthogonality constraints
  14. Mollifiers, adaptive sampling, and misfit switch
  15. Mollifiers
  16. ...and 23 more sections

Key Result

Theorem 3.2

Suppose $u$ is the solution to the $d$-dimensional wave equation eq:wave_eqn where $u_0 \in \mathcal{B}_1$ and $v_0 \in \mathcal{B}_0$. Then for the collection $\mathcal{M} := \{ u(\cdot, t) |_\Omega \mid t \in [0, T]\}$,

Figures (18)

  • Figure 1: Various solution manifolds of time-dependent problems $\mathcal{M} = \{ u(\cdot, t) \mid t \in [0, T]\}$ and their LRNR coefficient counts $\mathscr{C}$ (technical definition below in Def. \ref{['def:lrnr_cc']}). The solution of 1d and 2d wave equations, as well as the 1d and 2d advection equations, have a coefficient count of at most five (Thms. \ref{['thm:wave']} and \ref{['thm:advection']}). Solution manifolds of scalar conservation laws (e.g., the 1d Burgers' equation) have a coefficient count of at most twelve (Thm. \ref{['thm:claw']}).
  • Figure 2: A diagram depicting the hypernetwork and the low rank neural representation. The hypernetwork takes the time variable $t$ as input, and outputs the coefficients $\mathbf{s}$\ref{['eq:coeff']} which are used as the highlighted entries to generate the LRNR weights and biases. In this example, the input dimension is two, the output dimension is three, and the rank is three.
  • Figure 3: Plots of training data from the 1d Euler example (first row), 2d acoustics (second row), 2d Burgers (third row), and 2d advection (fourth row). For the 2d examples, AMR (Adaptive Mesh Refinement) data were used, where the finest-grid values at each spatial location were extracted as training data. The grid edges indicate the boundaries of AMR patches Berger1998. All data were generated using the Clawpack software package clawpack.
  • Figure 4: Comparison of reconstructions between SIREN and LRNR. Oscillations persist in the SIREN model trained with MSE loss, whereas the LRNR reconstruction exhibits damped oscillations and sharper jump discontinuities.
  • Figure 5: LRNR reconstruction of the solution to the 1d compressible Euler equations with periodic boundary conditions (Woodward-Colella shock problem Woodward1982Woodward1984LeVeque2002). (a,b) Zoom-in plots highlight the learned representation at interpolated times, $u(\cdot\,; f_{\text{hyper}}(t))$. In (a), the rarefaction waves and shocks that emerge after the shock collision are individually transported with high accuracy. In (b), the sharp peak formed at the shock merger is correctly placed and closely mimics the behavior of the finite volume solution, despite the fact that the precise timing of the collision must be inferred from data.
  • ...and 13 more figures

Theorems & Definitions (8)

  • Definition 2.1: Low rank neural representation (LRNR)
  • Definition 3.1: LRNR coefficient count
  • Theorem 3.2: Wave equation in $d$ dimensions
  • proof
  • Theorem 3.3: Advection equation in $d$ dimensions
  • proof
  • Theorem 3.4: 1$d$ scalar conservation laws
  • proof