The Role of Fibration Symmetries in Geometric Deep Learning

TL;DR

This work extends Geometric Deep Learning by introducing local fibration symmetries as a flexible inductive bias that relaxes the standard global symmetry assumption. The authors formalize a Fibration Test that upper-bounds the expressive power of GNNs and show this bound is tighter than the WL-based bound, reflecting the more permissive yet structured local symmetry. They demonstrate practical benefits by compressing graph-structured data via minimal fibration bases on QM9 and BRCA-TCGA without sacrificing performance, and by observing node synchronization in MLPs with a corresponding Fibration Gradient Descent that reduces network size while preserving accuracy. The framework generalizes beyond graphs to manifolds, bundles, and grids, offering a principled path to stronger generalization and computational efficiency in geometric learning through local symmetry induction.

Abstract

Geometric Deep Learning (GDL) unifies a broad class of machine learning techniques from the perspectives of symmetries, offering a framework for introducing problem-specific inductive biases like Graph Neural Networks (GNNs). However, the current formulation of GDL is limited to global symmetries that are not often found in real-world problems. We propose to relax GDL to allow for local symmetries, specifically fibration symmetries in graphs, to leverage regularities of realistic instances. We show that GNNs apply the inductive bias of fibration symmetries and derive a tighter upper bound for their expressive power. Additionally, by identifying symmetries in networks, we collapse network nodes, thereby increasing their computational efficiency during both inference and training of deep neural networks. The mathematical extension introduced here applies beyond graphs to manifolds, bundles, and grids for the development of models with inductive biases induced by local symmetries that can lead to better generalization.

Figures (6)

• Figure 1: Symmetries, inductive bias and expressive power. a) For each level of symmetries (i.e., fibrations, coverings, automorphisms), there is a partition of space of graphs $\mathcal{X}$ (see colored dashed lines). The partition induced by isomorphisms (e.g., 6 classes) is finer than the partition induced by coverings (e.g., 5 classes); and the latter is finer than that induced by the fibrations (e.g., 4 classes). b) Functions with inductive bias induced by isomorphisms have greater expressive power than those that satisfy the inductive bias induced by local isomorphisms (i.e., coverings) or induced by local in-isomorphisms (i.e. fibrations). GNNs are models with inductive bias induced by fibrations and the upper bound on its expressive power is determined by Fibration Test (see Lemma 1). c) The graphs $G_1$ and $G_2$ are not isomorphic but have the same fibration base. A map $F$ with inductive bias induced by isomorphisms (blue box) returns different outputs for the graphs ((i.e., $F(G_1) \neq F(G_2)$). On the other hand, a map $F$ with inductive bias induced by fibrations (yellow box) returns the same output for both graphs (i.e., $F(G_1)=F(G_2)$). d) Isomorphisms are global symmetries, while coverings and fibrations are local symmetries. Isomorphisms, coverings, and fibrations induce different inductive biases, which increase when the symmetry is weaker. This implies that expressive power is lesser for models that use weaker symmetries. e) In the Space of Mappings, there is a subspace of maps that satisfies the inductive bias induced by isomorphisms (blue area), coverings (red area), and fibrations (yellow area). GNNs are a subset of the maps with bias induced by fibrations (see green line).
• Figure 2: Fibration symmetries in datasets. Examples of the fibration symmetries observed in the graphs of QM9 and BRCA-TCGA datasets. a) Tricyclo [5.1.0.03,5] octane is a molecule composed of eight carbon atoms and twelve hydrogen atoms. The partition induced by automorphisms consists of six orbits, while the partition induced by fibrations consists of four fibers. One fiber consists of 4 hydrogen atoms (blue circles), one fiber of 8 hydrogen atoms (red circles), and two fibers of 4 carbon atoms (green and orange circles) b) Fibration base of Tricyclo octane. c) The biological interaction network in BRCA-TCGA consists of 9288 genes interacting in various ways represented by arrows of different colors. The genes CCR3, CCR4, CCR6, CCR7, and CXCR3 are in the same fiber because they have isomorphic input trees. The genes CCL5, CCL20, CCL19, and CXCL10 (resp. HECTD1, HECTD2, and HECTD3) are in the same fiber because they receive information from the same gene IL10 (resp., CDC34).
• Figure 3: Standard and Reduced form of GNNs. a) The structure of a GNN and the number of operations are derived from the structure of the input graphs $G$ (Standard form). Finding the fibration base $B$ for the input graph $G$ allows us to reduce the number of operations in GNNs (Reduced form). b) Loss function during the training stage for the task T1. c) Predictions of the network vs. dipole moment of the molecules. d) Comparison of the execution times for "standard form" and "reduced form" for task T1.
• Figure 4: Synchronicity in MLP. a) An MLP trained to classify images (e.g. digits in MNIST dataset). b) Evolution of MLP accuracy during the training stage for different datasets: MNIST, KMNIST, and Fashion. c) After the training stage, clusters of synchronized nodes are detected. The clusters depend on the data used $\mathcal{S}$. The results presented in the left (resp., right) column correspond to cases where $\mathcal{S}$ is the subset of samples from class 1 (resp., from all classes) within the test set. The distance matrix between nodes $\Lambda$ (see definition in Methods) is represented by a color map. The node indices are sorted by mean activity across all the samples in $\mathcal{S}$ (see Mean act). d) In the Size vs. Activity plane, we show the distribution of cluster size and activity across input images (i.e., $\mathcal{S}$ is a single image). In the upper (resp., right) panel, the marginal distribution of cluster activities (resp., sizes) is plotted.
• Figure 5: Fibration Gradient Descent (FB-GD). a) Usually, during the training stage of an MLP via gradient descent, the network's architecture remains unchanged over time (see Net 1 - Original DG). For FB-GD, both the connection weights and the network structure are updated (see FD-GB). Every $T$ epochs, clusters of nodes are detected to collapse them into a single node, similarly to constructing the bases (Net m+1) of a graph (Net m) with fibration symmetries $\Phi_{fib}$. b) Each cluster (colored circles) is associated with an activity interval of size $\epsilon$ (see color bar). The update equation for the parameters of the Network m+1 is designed to be compatible with the update equation for the previous Network m. c) The final size of the network (see Size L1/L2) and its performance (see Accuracy) depend on $\epsilon$ and $T$. In the limit $\epsilon \rightarrow 0$ and $T \rightarrow \infty$, the results of FB-GD coincide with those of the original gradient descent.

Theorems & Definitions (3)

• Theorem 1
• Theorem 2
• Theorem 3