Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

What Can We Learn From MIMO Graph Convolutions?

Andreas Roth, Thomas Liebig

TL;DR

The paper tackles the expressivity and efficiency gaps in applying graph convolutions to the MIMO setting by deriving a MIMO Graph Convolution ($MIMO\text{-}GC$) through the convolution theorem and introducing Localized MIMO Graph Convolutions (LMGCs) that operate on multiple computational graphs. It proves strong theoretical properties—injectivity on multisets for a single graph and linear independence when using multiple graphs—and demonstrates experimentally that LMGCs can unify superior aspects of GCN, GATv2, FAGCN, and GIN across graph-level and node-level tasks. The findings suggest LMGC as a powerful, scalable linear MPNN kernel capable of tailored multi-channel learning, with practical impact on molecular property prediction and heterogeneous graph tasks. Overall, the work provides a principled bridge between spectral, polynomial, and attention-based GNN approaches, enabling more flexible and expressive message passing via a multi-graph, multi-channel framework.

Abstract

Most graph neural networks (GNNs) utilize approximations of the general graph convolution derived in the graph Fourier domain. While GNNs are typically applied in the multi-input multi-output (MIMO) case, the approximations are performed in the single-input single-output (SISO) case. In this work, we first derive the MIMO graph convolution through the convolution theorem and approximate it directly in the MIMO case. We find the key MIMO-specific property of the graph convolution to be operating on multiple computational graphs, or equivalently, applying distinct feature transformations for each pair of nodes. As a localized approximation, we introduce localized MIMO graph convolutions (LMGCs), which generalize many linear message-passing neural networks. For almost every choice of edge weights, we prove that LMGCs with a single computational graph are injective on multisets, and the resulting representations are linearly independent when more than one computational graph is used. Our experimental results confirm that an LMGC can combine the benefits of various methods.

What Can We Learn From MIMO Graph Convolutions?

TL;DR

The paper tackles the expressivity and efficiency gaps in applying graph convolutions to the MIMO setting by deriving a MIMO Graph Convolution () through the convolution theorem and introducing Localized MIMO Graph Convolutions (LMGCs) that operate on multiple computational graphs. It proves strong theoretical properties—injectivity on multisets for a single graph and linear independence when using multiple graphs—and demonstrates experimentally that LMGCs can unify superior aspects of GCN, GATv2, FAGCN, and GIN across graph-level and node-level tasks. The findings suggest LMGC as a powerful, scalable linear MPNN kernel capable of tailored multi-channel learning, with practical impact on molecular property prediction and heterogeneous graph tasks. Overall, the work provides a principled bridge between spectral, polynomial, and attention-based GNN approaches, enabling more flexible and expressive message passing via a multi-graph, multi-channel framework.

Abstract

Most graph neural networks (GNNs) utilize approximations of the general graph convolution derived in the graph Fourier domain. While GNNs are typically applied in the multi-input multi-output (MIMO) case, the approximations are performed in the single-input single-output (SISO) case. In this work, we first derive the MIMO graph convolution through the convolution theorem and approximate it directly in the MIMO case. We find the key MIMO-specific property of the graph convolution to be operating on multiple computational graphs, or equivalently, applying distinct feature transformations for each pair of nodes. As a localized approximation, we introduce localized MIMO graph convolutions (LMGCs), which generalize many linear message-passing neural networks. For almost every choice of edge weights, we prove that LMGCs with a single computational graph are injective on multisets, and the resulting representations are linearly independent when more than one computational graph is used. Our experimental results confirm that an LMGC can combine the benefits of various methods.
Paper Structure (39 sections, 6 theorems, 36 equations, 5 figures, 9 tables)

This paper contains 39 sections, 6 theorems, 36 equations, 5 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Graph Convolutions
  4. MIMO Graph Convolution
  5. Computational Complexity
  6. Benefiting from the MIMO-GC
  7. Localized MIMO Graph Convolutions
  8. Theoretical Properties
  9. An LMGC Instantiation
  10. Related Work
  11. Graph Convolutions
  12. MIMO Improvements
  13. Approaches Related to the LMGC
  14. Experiments
  15. Methods
  16. ...and 24 more sections

Key Result

Theorem 1

Let ${\bm{X}}\in\mathbb{R}^{n\times d}$, ${\bm{\mathsf{\Theta}}}\in\mathbb{R}^{n\times c \times d}$, and the Fourier transform $F = {\bm{U}}^T\in\mathbb{R}^{n\times n}$ be given by the eigenvectors of the graph Laplacian ${\bm{\Lambda}}$. Then, their convolution is given as where ${\bm{A}}^{(k)} = {\bm{U}}_{:,k} ({\bm{U}}_{:,k})^T\in\mathbb{R}^{n\times n}$, ${\bm{W}}^{(k)} = \left(F({\bm{\mathsf{

Figures (5)

  • Figure 1: Connections between the graph convolution, polynomial filters, and message-passing approaches in the SISO and the MIMO case. Parts in yellow ( ) indicate existing contributions, parts in pink ( ) our contributions.
  • Figure 2: Examples of random filters $F({\bm{\theta}})$ in the graph Fourier domain.
  • Figure 3: Spectral filters based on Chebyshev polynomials $F({\bm{\theta}}) = \sum_{k=0}^K {w}_k T_k({\bm{\lambda}})$ of different degrees $K$.
  • Figure 4: Filters of the GCN for $F({\bm{\theta}}) = w{\bm{\lambda}}$ with different values for $w$.
  • Figure 5: Combination of $k$ repetitions of random GCN filters $F({\bm{\theta}}) = w_k{\bm{\lambda}} \odot \dots \odot w_1{\bm{\lambda}}$ with random values for $w_1,\dots,w_k$.

Theorems & Definitions (16)

  • Theorem 1: MIMO Graph Convolution (MIMO-GC)
  • Proposition 1: Universality of the MIMO-GC
  • Proposition 2: Every MIMO polynomial filter is a MIMO-GC with a specific filter
  • Example 1: GCN is a MIMO-GC
  • Definition 1
  • Example 2: GCN kipf2017semisupervised
  • Example 3: GAT veličković2018graph
  • Proposition 3: Injectivity
  • Proposition 4: Linear Independence
  • proof
  • ...and 6 more