The High Cost of Data Augmentation for Learning Equivariant Models

TL;DR

The paper analyzes how to enforce or approximate rotational symmetry in data-driven models by comparing exact symmetry via invariant architectures to two data augmentation schemes that approximate symmetry: quadrature-based augmentation over the Haar measure and random sampling. It proves that quadrature augmentation achieves exact invariance for polynomial models when the quadrature degree is sufficiently high, while random augmentation yields a near $T^{-1/2}$ convergence of the symmetry error. Through extensive 2D and 3D numerical experiments with $N=3$ particles, the authors show that quadrature augmentation typically outperforms random augmentation, particularly in pre-asymptotic regimes, though it incurs higher computational cost due to larger augmented LSQ systems. The work emphasizes that preserving symmetry can be crucial for conserving Noether quantities in simulations, and suggests that quadrature augmentation is a promising path toward exact symmetry in data-driven surrogate models, with potential implications for extending to nonlinear and deep learning settings.

Abstract

According to Noether's theorem the presence of a continuous symmetry in a Hamiltonian systems is equivalent to the existence of a conserved quantity, yet these symmetries are not always explicitly enforced in data-driven models. There remains a debate whether or not encoding of symmetry into a model architecture is the optimal approach. A competing approach is to target approximate symmetry through data augmentation. In this work, we study two approaches aimed at improving the symmetry properties of such an approximation scheme: one based on a quadrature rule for the Haar measure on the compact Lie group encoding the continuous symmetry of interest and one based on a random sampling of that Haar measure. We demonstrate both theoretically and empirically that the quadrature augmentation leads to exact symmetry preservation in polynomial models, while the random augmentation has only square-root convergence of the symmetrization error.

Figures (14)

• Figure 1: All computations are performed with a Verlet integrator of step-size $dt = 0.05$.
• Figure 2: An illustration of the distributions mentioned in Table \ref{['tab:1']}
• Figure 3: Approximation rates for the different distributions. Our sample size is 8000 and testset size is 2000. The design matrix for $\delta^*UU$ was found to be ill-conditioned, thus the LSQ required regularisation, which we performed by choosing a cutoff singular value of $10^{-4.5}$. This cutoff has been chosen as the one that minimises the approximation error. All three graphs share the same the y-axis. Remark that the blue, green and red plots are indistinguishable.
• Figure 4: $\varepsilon_{sym}$ for quadrature augmentation using $UUU$, $\delta UU$, $\delta^*UU$. The (unaugmented) training set size is 800 and the validation set size is 200. All figures share the same y-axis. The shaded lines correspond to the approximation error.
• Figure 5: $\varepsilon_{\mathrm{sym}}$ when no regularisation is applied for quadrature augmentation using $\delta^*UU$. The training set-size is 800 and the validation set-size is 200.

Theorems & Definitions (21)

• Definition 1
• Lemma 1
• proof
• Lemma 2
• proof
• Remark 1
• Lemma 3
• Lemma 4
• proof
• Lemma 5