Reservoir-Based Graph Convolutional Networks

Abstract

Message passing is a core mechanism in Graph Neural Networks (GNNs), enabling the iterative update of node embeddings by aggregating information from neighboring nodes. Graph Convolutional Networks (GCNs) exemplify this approach by adapting convolutional operations for graph structures, allowing features from adjacent nodes to be combined effectively. However, GCNs encounter challenges with complex or dynamic data. Capturing long-range dependencies often requires deeper layers, which not only increase computational costs but also lead to over-smoothing, where node embeddings become indistinguishable. To overcome these challenges, reservoir computing has been integrated into GNNs, leveraging iterative message-passing dynamics for stable information propagation without extensive parameter tuning. Despite its promise, existing reservoir-based models lack structured convolutional mechanisms, limiting their ability to accurately aggregate multi-hop neighborhood information. To address these limitations, we propose RGC-Net (Reservoir-based Graph Convolutional Network), which integrates reservoir dynamics with structured graph convolution. Key contributions include: (i) a reimagined convolutional framework with fixed random reservoir weights and a leaky integrator to enhance feature retention; (ii) a robust, adaptable model for graph classification; and (iii) an RGC-Net-powered transformer for graph generation with application to dynamic brain connectivity. Extensive experiments show that RGC-Net achieves state-of-the-art performance in classification and generative tasks, including brain graph evolution, with faster convergence and reduced over-smoothing. Source code is available at https://github.com/basiralab/RGC-Net .

Figures (9)

• Figure 1: Echo State Network (ESN) architecture with an input layer, a fixed-weight reservoir, and an output layer. Input weights $W_{\text{in}} \in \mathbb{R}^{n_{\text{res}} \times n_{\text{in}}}$ map the input vector $u(t) \in \mathbb{R}^{n_{\text{in}}}$ into the reservoir. The reservoir uses recurrent weights $W_{\text{res}} \in \mathbb{R}^{n_{\text{res}} \times n_{\text{res}}}$ to capture temporal dependencies. The output layer, with weights $W_{\text{out}} \in \mathbb{R}^{n_{\text{out}} \times n_{\text{res}}}$, generates the output vector $y(t+1) \in \mathbb{R}^{n_{\text{out}}}$, with only $W_{\text{out}}$ being trainable.
• Figure 2: Proposed RGC-Net architecture, comprising three components: a non-trainable input layer, a graph convolutional reservoir, and a trainable linear output layer.
• Figure 3: RGC-Net Architecture for Graph Classification. The input graph $G_{\text{inp}}$, defined by its adjacency matrix $A$ and node feature matrix $X$, is processed through RGC-Net. It begins with an Input Layer that transforms node features using fixed weights. This is followed by the Graph Convolutional Reservoir, where iterative self-iterations capture neighborhood information. The Linear Layer then projects these embeddings to a higher feature space, followed by Batch Normalization, ReLU, Average Global Pooling, and a Feed-Forward Network (FFN). The SoftMax layer produces the final output label $Y'$
• Figure 4: Architecture of the RGC-Net-Transformer model for temporal graph generation. The model employs RGC-Net layers within encoder and decoder networks to capture dynamic connectivity patterns and predict future graph structures. The encoder processes the input graph $G_{t_0}(A_{t_0}, X_{t_0})$ through Graph Attention (GAT), Batch Normalization, and RGC-Net layers, generating a latent representation $Z$. This is then decoded to reconstruct the predicted adjacency matrix $A_{t_1}$, utilizing reservoir-based layers to retain features across time steps.
• Figure 5: Connectivity distributions of connectomic datasets for brain graph evolution prediction across different time points. Figure (a) shows the EMCI-AD dataset with nearly the same connectivity distribution across time points. Figures (b) and (c) show connectivity distribution for simulated and SLIM160 datasets. The figures show that although the connectivity distribution between time points is quite similar, each time point slightly differs in the maximum density.

Theorems & Definitions (1)

• proof