Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Balancing Symmetry and Efficiency in Graph Flow Matching

Benjamin Honoré, Alba Carballo-Castro, Yiming Qin, Pascal Frossard

TL;DR

Experiments first show that symmetry-breaking can accelerate early training by providing an easier learning signal, but at the expense of encouraging shortcut solutions that can cause overfitting, where the model repeatedly generates graphs that are duplicates of the training set.

Abstract

Equivariance is central to graph generative models, as it ensures the model respects the permutation symmetry of graphs. However, strict equivariance can increase computational cost due to added architectural constraints, and can slow down convergence because the model must be consistent across a large space of possible node permutations. We study this trade-off for graph generative models. Specifically, we start from an equivariant discrete flow-matching model, and relax its equivariance during training via a controllable symmetry modulation scheme based on sinusoidal positional encodings and node permutations. Experiments first show that symmetry-breaking can accelerate early training by providing an easier learning signal, but at the expense of encouraging shortcut solutions that can cause overfitting, where the model repeatedly generates graphs that are duplicates of the training set. On the contrary, properly modulating the symmetry signal can delay overfitting while accelerating convergence, allowing the model to reach stronger performance with $19\%$ of the baseline training epochs.

Balancing Symmetry and Efficiency in Graph Flow Matching

TL;DR

Experiments first show that symmetry-breaking can accelerate early training by providing an easier learning signal, but at the expense of encouraging shortcut solutions that can cause overfitting, where the model repeatedly generates graphs that are duplicates of the training set.

Abstract

Equivariance is central to graph generative models, as it ensures the model respects the permutation symmetry of graphs. However, strict equivariance can increase computational cost due to added architectural constraints, and can slow down convergence because the model must be consistent across a large space of possible node permutations. We study this trade-off for graph generative models. Specifically, we start from an equivariant discrete flow-matching model, and relax its equivariance during training via a controllable symmetry modulation scheme based on sinusoidal positional encodings and node permutations. Experiments first show that symmetry-breaking can accelerate early training by providing an easier learning signal, but at the expense of encouraging shortcut solutions that can cause overfitting, where the model repeatedly generates graphs that are duplicates of the training set. On the contrary, properly modulating the symmetry signal can delay overfitting while accelerating convergence, allowing the model to reach stronger performance with of the baseline training epochs.
Paper Structure (37 sections, 11 equations, 11 figures, 2 tables)

This paper contains 37 sections, 11 equations, 11 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Assessing the relevance of positional encodings
  3. Mechanisms for symmetry modulation
  4. Symmetry breaking via scaled positional encodings.
  5. Symmetry restoration via in-training permutations.
  6. Dynamics optimization
  7. Evaluation metrics.
  8. Experimental results
  9. Effect of different positional encodings.
  10. Symmetry breaking-restoring cycle.
  11. Best performing configurations.
  12. Conclusion
  13. Related work
  14. Graph generative models
  15. Equivariant networks and symmetry breaking
  16. ...and 22 more sections

Figures (11)

  • Figure 1: Symmetry breaking restoring cycle. Starting from the initial point, equivariant paths converge slowly but preserve symmetry, non equivariant paths converge faster with a generalization gap, and the breaking restoring path achieves fast progress while recovering structural validity.
  • Figure 2: Comparison of the baseline setting and symmetry breaking sinusoidal positional encodings in (A), and with different invariance scalings $\lambda$ using (B) $\chi=\infty$ and (C) $\chi=10$. Panels show (a) Validity, (b) Novelty, (c) Uniqueness, and (d) VUN on the SBM dataset.
  • Figure 3: Comparison of DeFoG's baseline setting, with RRWP encodings and symmetry breaking sinusoidal PE with different invariance scaling $\lambda$ and time dependent permutation rates.
  • Figure 4: (A) Inverse of the time-dependent rate functions used in Figure \ref{['fig:timDep']}. (B) Phase space diagram of the aggregated metric UN, evaluated in the first $10000$ training steps.
  • Figure 5: Comparison of positional encodings with and without data augmentation (a) Validity (b) Novelty (c) Uniqueness (d) VUN, on the SBM dataset.
  • ...and 6 more figures