Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Noether's razor: Learning Conserved Quantities

Tycho F. A. van der Ouderaa, Mark van der Wilk, Pim de Haan

TL;DR

This work uses Noether's theorem to parameterise symmetries as learnable conserved quantities, which allows conserved quantities and associated symmetries to be learned directly from train data through approximate Bayesian model selection, jointly with the regular training procedure.

Abstract

Symmetries have proven useful in machine learning models, improving generalisation and overall performance. At the same time, recent advancements in learning dynamical systems rely on modelling the underlying Hamiltonian to guarantee the conservation of energy. These approaches can be connected via a seminal result in mathematical physics: Noether's theorem, which states that symmetries in a dynamical system correspond to conserved quantities. This work uses Noether's theorem to parameterise symmetries as learnable conserved quantities. We then allow conserved quantities and associated symmetries to be learned directly from train data through approximate Bayesian model selection, jointly with the regular training procedure. As training objective, we derive a variational lower bound to the marginal likelihood. The objective automatically embodies an Occam's Razor effect that avoids collapse of conservation laws to the trivial constant, without the need to manually add and tune additional regularisers. We demonstrate a proof-of-principle on $n$-harmonic oscillators and $n$-body systems. We find that our method correctly identifies the correct conserved quantities and U($n$) and SE($n$) symmetry groups, improving overall performance and predictive accuracy on test data.

Noether's razor: Learning Conserved Quantities

TL;DR

This work uses Noether's theorem to parameterise symmetries as learnable conserved quantities, which allows conserved quantities and associated symmetries to be learned directly from train data through approximate Bayesian model selection, jointly with the regular training procedure.

Abstract

Symmetries have proven useful in machine learning models, improving generalisation and overall performance. At the same time, recent advancements in learning dynamical systems rely on modelling the underlying Hamiltonian to guarantee the conservation of energy. These approaches can be connected via a seminal result in mathematical physics: Noether's theorem, which states that symmetries in a dynamical system correspond to conserved quantities. This work uses Noether's theorem to parameterise symmetries as learnable conserved quantities. We then allow conserved quantities and associated symmetries to be learned directly from train data through approximate Bayesian model selection, jointly with the regular training procedure. As training objective, we derive a variational lower bound to the marginal likelihood. The objective automatically embodies an Occam's Razor effect that avoids collapse of conservation laws to the trivial constant, without the need to manually add and tune additional regularisers. We demonstrate a proof-of-principle on -harmonic oscillators and -body systems. We find that our method correctly identifies the correct conserved quantities and U() and SE() symmetry groups, improving overall performance and predictive accuracy on test data.

Paper Structure

This paper contains 47 sections, 1 theorem, 26 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Hamiltonian mechanics
  4. Learning Hamiltonian mechanics from data
  5. Hamiltonian neural networks
  6. Noether's theorem
  7. Conserved quantity
  8. Symmetries generated by observables
  9. Automatic symmetry discovery
  10. Symmetrising Hamiltonians with Conserved Quantities
  11. Parameterising conserved quantities
  12. Symmetrising observables
  13. Automatic Symmetry Discovery using Noether's Razor
  14. Probabilistic model with symmetries embedded in the prior.
  15. Bayesian model selection for symmetry discovery
  16. ...and 32 more sections

Key Result

Theorem 1

The observable $O \in \mathcal{O}$ is a conserved quantity on the trajectories generated by Hamiltonian $H \in \mathcal{O}$ if and only if $H$ is invariant to $\mathcal{G}_O$, meaning that for all $\tau \in \mathbb{R}$, $H \circ \Phi^\tau_O=H$.

Figures (5)

  • Figure 1: Graphical probabilistic model. Trajectory data $X$ depends on a symmetrised Hamiltonian $H$ induced by non-symmetrised observable $F$ and conservation laws $C$.
  • Figure 2: Learned Hamiltonians on phase space of simple harmonic oscillator by HNN models.
  • Figure 3: Singular value and parallelness of the singular vectors of the learned generators, for $n$ oscilators. U($n$) is correctly learned.
  • Figure 4: Singular value and parallelness of the singular vectors of the learned generators for three body system in two dimensions. The 7-dimensional Lie group $\mathcal{G}$ of quadratic conserved quantities is correctly learned.
  • Figure 5: Learned generators associated by conserved quantities and their singular value decomposition. We find a subspace spanned by the 7 linear generators that correspond to the correct symmetries (see \ref{['sec:nbody-details']}): (1) rotation of the center of mass $R^{\mathrm{COM}}$, (2) rotation around the origin $R^{\mathrm{ABS}}$, (4+5) translation, (5+6+7) momentum-dependent translations $P, Q$, (8+9+10) inactive ($\lambda < 0.05$). The first 7 singular vectors lie in the ground truth subspace of generators with measured parallelness $||v_i^{||}|| > 0.95$.

Theorems & Definitions (2)

  • Theorem 1: Noether
  • proof