Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Symmetry Discovery for Different Data Types

Lexiang Hu, Yikang Li, Zhouchen Lin

TL;DR

LieSD is proposed, a method for discovering symmetries via trained neural networks which approximate the input-output mappings of the tasks and can accurately determine the number of Lie algebra bases without the need for expensive group sampling.

Abstract

Equivariant neural networks incorporate symmetries into their architecture, achieving higher generalization performance. However, constructing equivariant neural networks typically requires prior knowledge of data types and symmetries, which is difficult to achieve in most tasks. In this paper, we propose LieSD, a method for discovering symmetries via trained neural networks which approximate the input-output mappings of the tasks. It characterizes equivariance and invariance (a special case of equivariance) of continuous groups using Lie algebra and directly solves the Lie algebra space through the inputs, outputs, and gradients of the trained neural network. Then, we extend the method to make it applicable to multi-channel data and tensor data, respectively. We validate the performance of LieSD on tasks with symmetries such as the two-body problem, the moment of inertia matrix prediction, and top quark tagging. Compared with the baseline, LieSD can accurately determine the number of Lie algebra bases without the need for expensive group sampling. Furthermore, LieSD can perform well on non-uniform datasets, whereas methods based on GANs fail.

Symmetry Discovery for Different Data Types

TL;DR

LieSD is proposed, a method for discovering symmetries via trained neural networks which approximate the input-output mappings of the tasks and can accurately determine the number of Lie algebra bases without the need for expensive group sampling.

Abstract

Equivariant neural networks incorporate symmetries into their architecture, achieving higher generalization performance. However, constructing equivariant neural networks typically requires prior knowledge of data types and symmetries, which is difficult to achieve in most tasks. In this paper, we propose LieSD, a method for discovering symmetries via trained neural networks which approximate the input-output mappings of the tasks. It characterizes equivariance and invariance (a special case of equivariance) of continuous groups using Lie algebra and directly solves the Lie algebra space through the inputs, outputs, and gradients of the trained neural network. Then, we extend the method to make it applicable to multi-channel data and tensor data, respectively. We validate the performance of LieSD on tasks with symmetries such as the two-body problem, the moment of inertia matrix prediction, and top quark tagging. Compared with the baseline, LieSD can accurately determine the number of Lie algebra bases without the need for expensive group sampling. Furthermore, LieSD can perform well on non-uniform datasets, whereas methods based on GANs fail.

Paper Structure

This paper contains 32 sections, 8 theorems, 54 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Equivariant networks
  4. Symmetry discovery
  5. Background
  6. Lie group and Lie algebra
  7. Group representation
  8. Equivariance and invariance
  9. Measuring equivariance in neural networks
  10. Symmetry discovery for single-channel vector data
  11. Efficiently computing
  12. Extension to different data types
  13. Why the extension is necessary
  14. Extension to multi-channel data
  15. Extension to tensor data
  16. ...and 17 more sections

Key Result

Theorem 1

Given a simply connected Lie group $G$, its group representations on the input space $\mathcal{X} \subseteq \mathbb{R}^n$ and output space $\mathcal{Y} \subseteq \mathbb{R}^m$ are denoted respectively as $\rho_{\mathcal{X}}(g): G \rightarrow GL(n)$ and $\rho_{\mathcal{Y}}(g): G \rightarrow GL(m)$. T if and only if the corresponding Lie algebra bases of $\rho_{\mathcal{X}}(g)$ and $\rho_{\mathcal{Y

Figures (5)

  • Figure 1: The framework of LieSD. We first train a neural network on the task to approximate the input-output mapping, and then use its inputs, outputs, and gradients for symmetry discovery. Adapted to different data types, LieSD can solve the Lie algebra space, including the dimension of the Lie algebra space and the Lie algebra basis representations.
  • Figure 2: The visualization results of symmetry discovery in the two-body problem. (a): The top 6 smallest singular values solved by LieSD on the uniform dataset, which are arranged in descending order. (b): The Lie algebra basis corresponding to the minimum singular value solved by LieSD on the uniform dataset. (c): The Lie algebra basis learned by LieGAN with 1 channel on the uniform dataset. (d-f): Results corresponding to (a-c) on the non-uniform dataset.
  • Figure 3: The visualization results of symmetry discovery in predicting the moment of inertia matrix of LieSD. (a): The computed singular values, which are arranged in descending order. (b-d): Lie algebra bases corresponding to five nearly zero singular values in spaces $\mathcal{X}_i, \mathcal{Y}_1, \mathcal{Y}_2$, where basis $i$ corresponds to the singular value with index $i$.
  • Figure 4: The visualization results of the ablation study on symmetry discovery in predicting the moment of inertia matrix. (a): The computed singular values, which are arranged in descending order. (b-c): Lie algebra bases corresponding to the five smallest singular values in spaces $\mathcal{X}_i$ and $\mathcal{Y}$, where basis $i$ corresponds to the singular value with index $i$.
  • Figure 5: The visualization results of symmetry discovery in top quark tagging. (a): The singular values obtained by LieSD, which are sorted in descending order. (b): Lie algebra bases solved by LieSD, where basis $i$ corresponds to the $i$-th largest singular value. (c): The Lie algebra bases learned by LieGAN with 7 channels. (d): The Lie algebra bases learned by LieGAN with 9 channels.

Theorems & Definitions (12)

  • Theorem 1
  • Theorem 2
  • Theorem 3
  • Theorem 4
  • Theorem 4
  • proof
  • Theorem 4
  • proof
  • Theorem 4
  • proof
  • ...and 2 more