Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Neural Conjugate Flows: Physics-informed architectures with flow structure

Arthur Bizzi, Lucas Nissenbaum, João M. Pereira

TL;DR

It is demonstrated in numerical experiments how this topological group structure leads to concrete computational gains over other physics informed neural networks in estimating and extrapolating latent dynamics of ODEs, while training up to five times faster than other flow-based architectures.

Abstract

We introduce Neural Conjugate Flows (NCF), a class of neural network architectures equipped with exact flow structure. By leveraging topological conjugation, we prove that these networks are not only naturally isomorphic to a continuous group, but are also universal approximators for flows of ordinary differential equation (ODEs). Furthermore, topological properties of these flows can be enforced by the architecture in an interpretable manner. We demonstrate in numerical experiments how this topological group structure leads to concrete computational gains over other physics informed neural networks in estimating and extrapolating latent dynamics of ODEs, while training up to five times faster than other flow-based architectures.

Neural Conjugate Flows: Physics-informed architectures with flow structure

TL;DR

It is demonstrated in numerical experiments how this topological group structure leads to concrete computational gains over other physics informed neural networks in estimating and extrapolating latent dynamics of ODEs, while training up to five times faster than other flow-based architectures.

Abstract

We introduce Neural Conjugate Flows (NCF), a class of neural network architectures equipped with exact flow structure. By leveraging topological conjugation, we prove that these networks are not only naturally isomorphic to a continuous group, but are also universal approximators for flows of ordinary differential equation (ODEs). Furthermore, topological properties of these flows can be enforced by the architecture in an interpretable manner. We demonstrate in numerical experiments how this topological group structure leads to concrete computational gains over other physics informed neural networks in estimating and extrapolating latent dynamics of ODEs, while training up to five times faster than other flow-based architectures.

Paper Structure

This paper contains 26 sections, 2 theorems, 46 equations, 12 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Our contributions
  3. Flows and conjugation
  4. Flows
  5. Topological Conjugation
  6. Affine Flows and Universality
  7. Neural Conjugate Flows
  8. Neural conjugation
  9. The network $\mathcal{H}$
  10. The flow $\Psi$
  11. Numerical Experiments
  12. Forward Problem: FitzHugh-Nagumo
  13. Inverse Problem: Hodgkin-Huxley
  14. Analysis and Additional Results
  15. Final Remarks
  16. ...and 11 more sections

Key Result

Theorem 1

Let $F:\mathbb{R}^n\mapsto \mathbb{R}^n$ be a Lipschitz-continuous vector field. Then for any positive integer $m$ there exists an augmentation $\mathbf{a} \in \mathbb{R}^m$, a component $G: \mathbb{R}^n\times\mathbb{R}^m \mapsto \mathbb{R}^m$, a matrix $\hat{\mathbf{A}} \in \mathbb{R}^{(n+m)\times( is conjugated to the affine system:

Figures (12)

  • Figure 1: Spurious convergence of an MLP to equilibrium, denoting lack of uniqueness (see the Appendix for experimental details). The tighter structure of NCFs makes this less likely.
  • Figure 2: Extrapolation capacity of NCF versus an MLP (see Section 4 for details). The NCF is able to generalize beyond the training time of $10$ms, the MLP is not.
  • Figure 3: Deforming a harmonic oscillator's orbit to match the limit cycle of a Van-der-Pol system.
  • Figure 4: The NCF pipeline: Change variables to the conjugate manifold, iterate, then change back.
  • Figure 5: A standard coupling layer ensemble in $\mathbb{R}^2$. Inverse evaluation may be done by reversing the arrows and replacing $+$ for $-$. Notice that each MLP only 'sees' half the input, leading to reduced representation power.
  • ...and 7 more figures

Theorems & Definitions (4)

  • Theorem 1
  • Definition 1: Neural Conjugate Flow
  • Theorem 2: Universal Approximation of Affine NCFs
  • proof : Proof of Theorem 1