Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Group Averaging for Physics Applications: Accuracy Improvements at Zero Training Cost

Valentino F. Foit, David W. Hogg, Soledad Villar

TL;DR

This work targets the gap between theory and practice for symmetry-preserving emulators in physics by proposing post-training group averaging to enforce exact equivariance with zero training cost. It uses the Reynolds operator, $(\mathcal{Q} f)(x) = \int_G \psi(g^{-1}) f(\phi(g)x) \, d\lambda(g)$, to average predictions over a symmetry group, approximated discretely for small groups or via Monte Carlo for continuous ones. Empirical results on the Well physics datasets show substantial accuracy gains, up to $37\%$ in VRMSE, across multiple systems and time horizons, while preserving arbitrary model architectures and training regimes. The findings advocate for routinely applying group averaging as a simple, effective technique to impose exact symmetries in physics-informed predictive models, with extensions like frame averaging and considerations for non-compact groups outlined for future work.

Abstract

Many machine learning tasks in the natural sciences are precisely equivariant to particular symmetries. Nonetheless, equivariant methods are often not employed, perhaps because training is perceived to be challenging, or the symmetry is expected to be learned, or equivariant implementations are seen as hard to build. Group averaging is an available technique for these situations. It happens at test time; it can make any trained model precisely equivariant at a (often small) cost proportional to the size of the group; it places no requirements on model structure or training. It is known that, under mild conditions, the group-averaged model will have a provably better prediction accuracy than the original model. Here we show that an inexpensive group averaging can improve accuracy in practice. We take well-established benchmark machine learning models of differential equations in which certain symmetries ought to be obeyed. At evaluation time, we average the models over a small group of symmetries. Our experiments show that this procedure always decreases the average evaluation loss, with improvements of up to 37\% in terms of the VRMSE. The averaging produces visually better predictions for continuous dynamics. This short paper shows that, under certain common circumstances, there are no disadvantages to imposing exact symmetries; the ML4PS community should consider group averaging as a cheap and simple way to improve model accuracy.

Group Averaging for Physics Applications: Accuracy Improvements at Zero Training Cost

TL;DR

This work targets the gap between theory and practice for symmetry-preserving emulators in physics by proposing post-training group averaging to enforce exact equivariance with zero training cost. It uses the Reynolds operator, , to average predictions over a symmetry group, approximated discretely for small groups or via Monte Carlo for continuous ones. Empirical results on the Well physics datasets show substantial accuracy gains, up to in VRMSE, across multiple systems and time horizons, while preserving arbitrary model architectures and training regimes. The findings advocate for routinely applying group averaging as a simple, effective technique to impose exact symmetries in physics-informed predictive models, with extensions like frame averaging and considerations for non-compact groups outlined for future work.

Abstract

Many machine learning tasks in the natural sciences are precisely equivariant to particular symmetries. Nonetheless, equivariant methods are often not employed, perhaps because training is perceived to be challenging, or the symmetry is expected to be learned, or equivariant implementations are seen as hard to build. Group averaging is an available technique for these situations. It happens at test time; it can make any trained model precisely equivariant at a (often small) cost proportional to the size of the group; it places no requirements on model structure or training. It is known that, under mild conditions, the group-averaged model will have a provably better prediction accuracy than the original model. Here we show that an inexpensive group averaging can improve accuracy in practice. We take well-established benchmark machine learning models of differential equations in which certain symmetries ought to be obeyed. At evaluation time, we average the models over a small group of symmetries. Our experiments show that this procedure always decreases the average evaluation loss, with improvements of up to 37\% in terms of the VRMSE. The averaging produces visually better predictions for continuous dynamics. This short paper shows that, under certain common circumstances, there are no disadvantages to imposing exact symmetries; the ML4PS community should consider group averaging as a cheap and simple way to improve model accuracy.

Paper Structure

This paper contains 13 sections, 12 equations, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. Our contributions
  3. Methods
  4. Surrogate models
  5. Equivariance from group averaging
  6. Numerical experiments
  7. Discussion
  8. Related work
  9. Groups
  10. Dihedral groups
  11. Lie groups
  12. A vector is something that transforms like a vector
  13. Experimental details

Figures (1)

  • Figure 1: Example comparisons of time evolutions of the concentration of two datasets. \ref{['active_matter_sim']} After $t=10$ steps we visually start to observe the benefits of group averaging. The equivariant model has a lower loss than the baseline at all times and is closer to the simulation (truth). At later times significant deviations between the approaches become apparent. \ref{['gray_scott_sim']} The equivariant model captures the features of the simulation relatively well for long times, while the baseline model develops several fictitious clusters.