Cooperative Sheaf Neural Networks

TL;DR

This work identifies a fundamental limitation in undirected sheaf-based diffusion: the lack of node-level cooperative communication (listen/propagate). By introducing directed cellular sheaves and their in-/out-degree Laplacians, the authors formulate Cooperative Sheaf Neural Networks (CSNNs) that learn per-node source and target maps to enable customizable information diffusion. The key theoretical results show CSNNs can realize distant, selective communication and expand the effective receptive field to promote long-range information flow, mitigating oversquashing. Empirically, CSNNs outperform prior sheaf-based and cooperative GNN approaches on heterophilic node classification and long-range graph tasks, while maintaining competitive efficiency. These findings suggest direction-aware, cooperative diffusion as a powerful paradigm for robust graph representation learning.

Abstract

Sheaf diffusion has recently emerged as a promising design pattern for graph representation learning due to its inherent ability to handle heterophilic data and avoid oversmoothing. Meanwhile, cooperative message passing has also been proposed as a way to enhance the flexibility of information diffusion by allowing nodes to independently choose whether to propagate/gather information from/to neighbors. A natural question ensues: is sheaf diffusion capable of exhibiting this cooperative behavior? Here, we provide a negative answer to this question. In particular, we show that existing sheaf diffusion methods fail to achieve cooperative behavior due to the lack of message directionality. To circumvent this limitation, we introduce the notion of cellular sheaves over directed graphs and characterize their in- and out-degree Laplacians. We leverage our construction to propose Cooperative Sheaf Neural Networks (CSNNs). Theoretically, we characterize the receptive field of CSNN and show it allows nodes to selectively attend (listen) to arbitrarily far nodes while ignoring all others in their path, potentially mitigating oversquashing. Our experiments show that CSNN presents overall better performance compared to prior art on sheaf diffusion as well as cooperative graph neural networks.

Figures (6)

• Figure 1: On the left, a cellular sheaf shown for a single edge of an undirected graph with stalks isomorphic to $\mathbb{R}^2$. The restriction maps $\mathcal{F}_{i \unlhd e}, \mathcal{F}_{j \unlhd e}$ move the vector features between these spaces. On the right, the analogous situation for a sheaf on a single pair of directed edges. Then there are four, possibly distinct, restriction maps $\mathcal{F}_{i \unlhd ij},\mathcal{F}_{i \unlhd ji},\mathcal{F}_{j \unlhd ij}, \mathcal{F}_{j \unlhd ji}$.
• Figure 2: ${\mathbf{O}}_i = 0$ creates the effect of isolating the node $i$. Directed edges provide the possibility of performing LISTEN and PROPAGATE, separately.
• Figure 3: Given a graph $(a)$ we illustrate the consequences of preventing node $3$ from listening. For NSD $(b)$, this means $L_{\mathcal{F}}(\mathbf{X})_3$ must not depend on $\mathbf{x_j},$ for $j=1,2,4$ implying $\mathcal{F}_{j \unlhd e} = 0$ and leading to $L_{\mathcal{F}}(\mathbf{X})_j$ not depending on $\mathbf{x_3}$, preventing node $3$ from propagating information. In CSNN $(c)$, we can set ${\mathbf{T}}_3 = 0$. Provided ${\mathbf{S}}_3 \neq 0$, outbound communication is possible.
• Figure 4: Illustration of \ref{['ex:oversquashing']}. At layer $t$, we consider that all maps but ${\mathbf{T}}_{4 - t, t}$ and ${\mathbf{S}}_{4 - (t - 1), t}$ are $0$, enabling the flow of information from right to left following the bottom edges.
• Figure 5: Accuracy for increasing tree depths in the NeighborsMatch task. CSNN consistently achieves 100% accuracy for all values of $r$.

Theorems & Definitions (14)

• Definition 2.1
• Definition 2.2
• Definition 2.3
• Proposition 3.1
• Definition 3.2
• Definition 3.3
• Proposition 4.1
• Proposition 4.2
• Proposition 4.3
• Example 4.4