Heterogeneous Graph Alignment for Joint Reasoning and Interpretability

TL;DR

MGMT tackles the challenge of integrating heterogeneous graphs with unaligned node sets by coupling per-graph Graph Transformer encoders with a depth-aware aggregation scheme and building an explicit meta-graph over attention-selected supernodes. The meta-graph enables fine-grained cross-graph message passing while preserving intra-graph topology, yielding improved graph-level predictions and interpretable substructure alignments. Theoretical results show MGMT can represent $L$-hop mixing and achieves smaller approximation error than late fusion, and empirical evaluations on synthetic and neuroscience datasets demonstrate consistent gains and informative substructure explanations. The approach offers a principled, backbone-agnostic framework for multi-graph reasoning with practical impact in domains like neuroscience and biomedical data analysis.

Abstract

Multi-graph learning is crucial for extracting meaningful signals from collections of heterogeneous graphs. However, effectively integrating information across graphs with differing topologies, scales, and semantics, often in the absence of shared node identities, remains a significant challenge. We present the Multi-Graph Meta-Transformer (MGMT), a unified, scalable, and interpretable framework for cross-graph learning. MGMT first applies Graph Transformer encoders to each graph, mapping structure and attributes into a shared latent space. It then selects task-relevant supernodes via attention and builds a meta-graph that connects functionally aligned supernodes across graphs using similarity in the latent space. Additional Graph Transformer layers on this meta-graph enable joint reasoning over intra- and inter-graph structure. The meta-graph provides built-in interpretability: supernodes and superedges highlight influential substructures and cross-graph alignments. Evaluating MGMT on both synthetic datasets and real-world neuroscience applications, we show that MGMT consistently outperforms existing state-of-the-art models in graph-level prediction tasks while offering interpretable representations that facilitate scientific discoveries. Our work establishes MGMT as a unified framework for structured multi-graph learning, advancing representation techniques in domains where graph-based data plays a central role.

Figures (8)

• Figure 1: Architecture of Multi-Graph Meta-Transformer (MGMT). Depth-Aware GT layers process individual graphs, extracting supernodes to form a meta-graph. Additional GT layers model both intra- and inter-graph interactions.
• Figure 2: a. Average test accuracy and standard error bars (computed over 50 repetitions) on three synthetic datasets. In all experiments, each sample consists of five synthetic graphs, which we refer to as Modalities 1–5. Experiment 1 (Setting 1) uses 100 samples, with 5 nodes per graph, all of which are informative. Experiments 2 and 3 (Setting 2) both involve structured noise: Experiment 2 uses 100 samples and Experiment 3 uses 2,000 samples; in both, each graph has 50 nodes, of which 40 are informative. The proposed MGMT model achieves the best overall performance. b. Test accuracies of baseline models on the LFP and Alzheimer’s disease datasets. Each bar represents the average test accuracy across 5 folds, along with the corresponding standard error. In both applications, MGMT consistently outperforms all other models.
• Figure 3: Cross-animal supernode and edge frequency map from MGMT. Each dashed box corresponds to one rat; node size and color indicate supernode selection frequency, and line color reflects edge occurrence frequency. High-frequency supernodes and edges cluster in distal CA1 (right side), with cross-rat superedges primarily linking distal regions across animals. Mitt exhibits weaker connectivity.
• Figure 4: Layer-wise attention patterns for the LFP data (SuperChris). Each panel shows the same subject-level LFP connectivity graph, along with the learned depth-confidence scores $\Gamma_{\ell}$ for each Transformer layer $\ell$, as well as the corresponding edge-level attention scores and node-wise summed attention weights (with warmer colors indicating higher attention or summed weights).
• Figure A5: Scalability analysis of MGMT with respect to key input parameters. We evaluate the empirical runtime of MGMT under controlled variations of (i) number of nodes per graph ($N$), (ii) number of graphs per sample ($n$), (iii) number of samples (log scale), and (iv) feature dimensionality ($d$).Runtime scales quadratically with $N$ due to the dense self-attention in the graph-specific Graph Transformers ($\mathcal{O}(N^2 \cdot d)$), and linearly with $n$, confirming the modular and scalable design of MGMT. Sample size and feature dimension contribute to runtime growth in accordance with expectations, with minor deviations at small scales. Linear and quadratic regression fits are shown for interpretability, along with corresponding $R^2$ values.

Theorems & Definitions (13)

• Definition 4.1: $L$-hop mixing with general message passing
• Remark 4.2
• Theorem 4.3
• Theorem 4.4
• proof : Proof of \ref{['thm:mgmt-lhop']}
• Remark A1
• proof : Proof of Theorem \ref{['thm:late-fusion']}
• Theorem A1
• proof
• Proposition A2