Learning Infinitesimal Generators of Continuous Symmetries from Data

TL;DR

This work tackles the problem of discovering continuous symmetries from data without assuming predefined Lie groups. It introduces a framework that models one-parameter groups via Neural ODEs, parameterizing infinitesimal generators with an MLP and guiding learning through a differentiable validity score applied to transformed data from images and PDEs. The approach uncovers both affine and non-affine symmetries, enabling automatic data augmentation that improves downstream tasks such as image classification and neural operator learning for PDEs, even with limited data. Empirical results show alignment of learned vectors with the affine basis on CIFAR-10, identification of non-affine or approximate PDE symmetries, and meaningful performance gains, highlighting the method's potential to broaden symmetry-learning beyond traditional, manually specified groups.

Abstract

Exploiting symmetry inherent in data can significantly improve the sample efficiency of a learning procedure and the generalization of learned models. When data clearly reveals underlying symmetry, leveraging this symmetry can naturally inform the design of model architectures or learning strategies. Yet, in numerous real-world scenarios, identifying the specific symmetry within a given data distribution often proves ambiguous. To tackle this, some existing works learn symmetry in a data-driven manner, parameterizing and learning expected symmetry through data. However, these methods often rely on explicit knowledge, such as pre-defined Lie groups, which are typically restricted to linear or affine transformations. In this paper, we propose a novel symmetry learning algorithm based on transformations defined with one-parameter groups, continuously parameterized transformations flowing along the directions of vector fields called infinitesimal generators. Our method is built upon minimal inductive biases, encompassing not only commonly utilized symmetries rooted in Lie groups but also extending to symmetries derived from nonlinear generators. To learn these symmetries, we introduce a notion of a validity score that examine whether the transformed data is still valid for the given task. The validity score is designed to be fully differentiable and easily computable, enabling effective searches for transformations that achieve symmetries innate to the data. We apply our method mainly in two domains: image data and partial differential equations, and demonstrate its advantages. Our codes are available at \url{https://github.com/kogyeonghoon/learning-symmetry-from-scratch.git}.

Figures (15)

• Figure 1: (a) Examples of the vector fields. $V_3$ is a learned symmetry which is approximately a rotation, while $V_7$ is not a symmetry, thus having a high validity score. (b) Transformed CIFAR-10 images using the learned generators. All the vector fields and transformations learned from CIFAR-10 are presented in \ref{['fig:img_vis']} of \ref{['sec:appendix:expresults']}. (c) Transformation of PDEs (KS equation) with learned symmetries: time translation (t-tsl) and Galilean boost (gal).
• Figure 2: An example of a flow.
• Figure 3: Process of learning symmetry.
• Figure 4: (a) Self inner-products of the learned generators. (b) Inner product comparison of the learned generators with the affine generators. (c) Affine-ness of learned generators.
• Figure 5: Inner products between the learned non-affine symmetry generators and the ground truth. The results including the affine symmetry generators are shown in \ref{['fig:pde_ip1']}.