Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

When GNNs meet symmetry in ILPs: an orbit-based feature augmentation approach

Qian Chen, Lei Li, Qian Li, Jianghua Wu, Akang Wang, Ruoyu Sun, Xiaodong Luo, Tsung-Hui Chang, Qingjiang Shi

TL;DR

This work tackles the challenge of symmetry in ILPs where variables are permuted without changing problem structure, which hampers GNN-based prediction. It shows that the combination of formulation symmetry with permutation equivariance/invariance causes symmetric variables to be indistinguishable by classic GNNs. The authors propose three guiding principles for feature augmentation and introduce an orbit-based augmentation scheme (Orbit, Orbit+) that groups symmetric variables into orbits and samples augmented features per orbit to break symmetry efficiently. Empirical results on three symmetric ILP benchmarks demonstrate improved predictive accuracy and training efficiency, validating the symmetry-aware augmentation approach and its parsimony advantages. The work highlights both the practicality of symmetry-aware GNN improvements and the need for careful consideration of isomorphic consistency in supervised settings.

Abstract

A common characteristic in integer linear programs (ILPs) is symmetry, allowing variables to be permuted without altering the underlying problem structure. Recently, GNNs have emerged as a promising approach for solving ILPs. However, a significant challenge arises when applying GNNs to ILPs with symmetry: classic GNN architectures struggle to differentiate between symmetric variables, which limits their predictive accuracy. In this work, we investigate the properties of permutation equivariance and invariance in GNNs, particularly in relation to the inherent symmetry of ILP formulations. We reveal that the interaction between these two factors contributes to the difficulty of distinguishing between symmetric variables. To address this challenge, we explore the potential of feature augmentation and propose several guiding principles for constructing augmented features. Building on these principles, we develop an orbit-based augmentation scheme that first groups symmetric variables and then samples augmented features for each group from a discrete uniform distribution. Empirical results demonstrate that our proposed approach significantly enhances both training efficiency and predictive performance.

When GNNs meet symmetry in ILPs: an orbit-based feature augmentation approach

TL;DR

This work tackles the challenge of symmetry in ILPs where variables are permuted without changing problem structure, which hampers GNN-based prediction. It shows that the combination of formulation symmetry with permutation equivariance/invariance causes symmetric variables to be indistinguishable by classic GNNs. The authors propose three guiding principles for feature augmentation and introduce an orbit-based augmentation scheme (Orbit, Orbit+) that groups symmetric variables into orbits and samples augmented features per orbit to break symmetry efficiently. Empirical results on three symmetric ILP benchmarks demonstrate improved predictive accuracy and training efficiency, validating the symmetry-aware augmentation approach and its parsimony advantages. The work highlights both the practicality of symmetry-aware GNN improvements and the need for careful consideration of isomorphic consistency in supervised settings.

Abstract

A common characteristic in integer linear programs (ILPs) is symmetry, allowing variables to be permuted without altering the underlying problem structure. Recently, GNNs have emerged as a promising approach for solving ILPs. However, a significant challenge arises when applying GNNs to ILPs with symmetry: classic GNN architectures struggle to differentiate between symmetric variables, which limits their predictive accuracy. In this work, we investigate the properties of permutation equivariance and invariance in GNNs, particularly in relation to the inherent symmetry of ILP formulations. We reveal that the interaction between these two factors contributes to the difficulty of distinguishing between symmetric variables. To address this challenge, we explore the potential of feature augmentation and propose several guiding principles for constructing augmented features. Building on these principles, we develop an orbit-based augmentation scheme that first groups symmetric variables and then samples augmented features for each group from a discrete uniform distribution. Empirical results demonstrate that our proposed approach significantly enhances both training efficiency and predictive performance.
Paper Structure (33 sections, 3 theorems, 7 equations, 4 figures, 6 tables, 1 algorithm)

This paper contains 33 sections, 3 theorems, 7 equations, 4 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. ILP and symmetry
  4. Learning tasks
  5. Bipartite representation
  6. Issues occur when GNNs meet formulation symmetry
  7. Methodology
  8. Principles for constructing augmented features
  9. Orbit-based feature augmentation
  10. Analysis
  11. Random noise from a uniform distribution (Uniform)
  12. Positional IDs (Position)
  13. On the principles of distinguishability and isomorphic consistency
  14. On the principle of augmentation parsimony
  15. Experiments
  16. ...and 18 more sections

Key Result

Proposition 1

Under Assumption assump:permutation_eq_inv, if a permutation $\pi \in S_n$ is a formulation symmetry of (ILP), then we have $f_\theta({\mathcal{A}})_i = f_\theta({\mathcal{A}})_{\pi(i)}$. Further, the elements of $f_\theta({\mathcal{A}})$ correspond to the same orbit are identical, i.e., $f_\theta({

Figures (4)

  • Figure 1: Left: An ILP instance where $x_1$ and $x_2$ are symmetric. Middle: A bipartite representation of the ILP instance. The variable and constraint nodes correspond to their counterparts in the ILP instance, with edges connecting them denoting the coefficients of variables in constraints. Right: The outputs for the symmetric variables are identical due to symmetry, thus GNNs cannot correctly predict the optimal solution.
  • Figure 2: Validation losses of different schemes.
  • Figure : Orbit-based feature augmentation
  • Figure : Statistics about the datasets.

Theorems & Definitions (7)

  • Definition 1
  • Definition 2
  • Definition 3
  • Proposition 1
  • Corollary 1
  • Proposition 2
  • Example 1