Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GRAMA: Adaptive Graph Autoregressive Moving Average Models

Moshe Eliasof, Alessio Gravina, Andrea Ceni, Claudio Gallicchio, Davide Bacciu, Carola-Bibiane Schönlieb

TL;DR

GRAMA addresses oversquashing and long-range dependencies in graph neural networks by transforming static graphs into sequences and applying an adaptive ARMA mechanism with selective coefficient learning, preserving permutation equivariance. The authors establish a theoretical equivalence between GRAMA and graph-based State Space Models, derive stability and long-range propagation conditions, and show empirical gains across 14 datasets using multiple backbones. The results demonstrate that GRAMA improves long-range information exchange over standard MPNNs and remains competitive with Graph Transformers and Graph SSMS, while maintaining scalability. This work provides a principled, flexible framework bridging sequential models and graph learning for enhanced long-distance propagation in graphs.

Abstract

Graph State Space Models (SSMs) have recently been introduced to enhance Graph Neural Networks (GNNs) in modeling long-range interactions. Despite their success, existing methods either compromise on permutation equivariance or limit their focus to pairwise interactions rather than sequences. Building on the connection between Autoregressive Moving Average (ARMA) and SSM, in this paper, we introduce GRAMA, a Graph Adaptive method based on a learnable Autoregressive Moving Average (ARMA) framework that addresses these limitations. By transforming from static to sequential graph data, GRAMA leverages the strengths of the ARMA framework, while preserving permutation equivariance. Moreover, GRAMA incorporates a selective attention mechanism for dynamic learning of ARMA coefficients, enabling efficient and flexible long-range information propagation. We also establish theoretical connections between GRAMA and Selective SSMs, providing insights into its ability to capture long-range dependencies. Extensive experiments on 14 synthetic and real-world datasets demonstrate that GRAMA consistently outperforms backbone models and performs competitively with state-of-the-art methods.

GRAMA: Adaptive Graph Autoregressive Moving Average Models

TL;DR

GRAMA addresses oversquashing and long-range dependencies in graph neural networks by transforming static graphs into sequences and applying an adaptive ARMA mechanism with selective coefficient learning, preserving permutation equivariance. The authors establish a theoretical equivalence between GRAMA and graph-based State Space Models, derive stability and long-range propagation conditions, and show empirical gains across 14 datasets using multiple backbones. The results demonstrate that GRAMA improves long-range information exchange over standard MPNNs and remains competitive with Graph Transformers and Graph SSMS, while maintaining scalability. This work provides a principled, flexible framework bridging sequential models and graph learning for enhanced long-distance propagation in graphs.

Abstract

Graph State Space Models (SSMs) have recently been introduced to enhance Graph Neural Networks (GNNs) in modeling long-range interactions. Despite their success, existing methods either compromise on permutation equivariance or limit their focus to pairwise interactions rather than sequences. Building on the connection between Autoregressive Moving Average (ARMA) and SSM, in this paper, we introduce GRAMA, a Graph Adaptive method based on a learnable Autoregressive Moving Average (ARMA) framework that addresses these limitations. By transforming from static to sequential graph data, GRAMA leverages the strengths of the ARMA framework, while preserving permutation equivariance. Moreover, GRAMA incorporates a selective attention mechanism for dynamic learning of ARMA coefficients, enabling efficient and flexible long-range information propagation. We also establish theoretical connections between GRAMA and Selective SSMs, providing insights into its ability to capture long-range dependencies. Extensive experiments on 14 synthetic and real-world datasets demonstrate that GRAMA consistently outperforms backbone models and performs competitively with state-of-the-art methods.
Paper Structure (35 sections, 5 theorems, 23 equations, 3 figures, 17 tables)

This paper contains 35 sections, 5 theorems, 23 equations, 3 figures, 17 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. GRAMA
  4. Graph Neural ARMA
  5. Learning Adaptive Graph ARMA Coefficients
  6. Theoretical Properties of GRAMA
  7. Experiments
  8. Graph Feature Transfer
  9. Graph Property Prediction
  10. Long-Range Benchmark
  11. GNN Benchmarks
  12. Conclusion
  13. Additional Related Work
  14. Proofs
  15. Proof of \ref{['thm:equivalence']}
  16. ...and 20 more sections

Key Result

Theorem 4.1

For every ARMA model, there exists an equivalent State Space Model (SSM) representation, and conversely, for every linear SSM, there exists an equivalent ARMA model representation.

Figures (3)

  • Figure 1: An illustration of the GRAMA framework with $R{=}L$ recurrences. We embed a static input graph into a sequence of graphs. This sequence is the input for the first GRAMA block. A GRAMA block is composed of a neural ARMA$(p,q)$ layer with adaptive autoregressive $\phi= \{\phi_i\}_{i=1}^{p}$ and moving average $\theta = \{\theta_j\}_{j=1}^{q}$ coefficients, and a graph-informed residual update via a GNN backbone. A GRAMA block transforms a graph sequence into a sequence. Each GRAMA block is a linear system, and non-linearities are applied between GRAMA blocks, as in \ref{['eq:sTH_grama_block']}.
  • Figure 2: Feature transfer performance on (a) Line, (b) Ring, and (c) Crossed-Ring graphs.
  • Figure 3: Line, ring, and crossed-ring graphs where the distance between source and target nodes is equal to 5. Nodes marked with "S" are source nodes, while the nodes with a "T" are target nodes.

Theorems & Definitions (10)

  • Theorem 4.1: Equivalence between ARMA models and State Space Models
  • Lemma 4.2: Stability of GRAMA
  • Theorem 4.3: Sufficient condition for GRAMA stability
  • Theorem 4.4: GRAMA allows long-range interactions
  • proof
  • proof
  • proof
  • Lemma B.1: Long-range interactions in GRAMA
  • proof
  • proof