Out-of-Distribution Generalized Dynamic Graph Neural Network with Disentangled Intervention and Invariance Promotion

TL;DR

This work proposes Disentangled Intervention-based Dynamic graph Attention networks with Invariance Promotion (I-DIDA) to handle spatio-temporal distribution shifts in dynamic graphs by discovering and utilizing invariant patterns, i.e., structures and features whose predictive abilities are stable across distribution shifts.

Abstract

Dynamic graph neural networks (DyGNNs) have demonstrated powerful predictive abilities by exploiting graph structural and temporal dynamics. However, the existing DyGNNs fail to handle distribution shifts, which naturally exist in dynamic graphs, mainly because the patterns exploited by DyGNNs may be variant with respect to labels under distribution shifts. In this paper, we propose Disentangled Intervention-based Dynamic graph Attention networks with Invariance Promotion (I-DIDA) to handle spatio-temporal distribution shifts in dynamic graphs by discovering and utilizing invariant patterns, i.e., structures and features whose predictive abilities are stable across distribution shifts. Specifically, we first propose a disentangled spatio-temporal attention network to capture the variant and invariant patterns. By utilizing the disentangled patterns, we design a spatio-temporal intervention mechanism to create multiple interventional distributions and an environment inference module to infer the latent spatio-temporal environments, and minimize the variance of predictions among these intervened distributions and environments, so that our model can make predictions based on invariant patterns with stable predictive abilities under distribution shifts. Extensive experiments demonstrate the superiority of our method over state-of-the-art baselines under distribution shifts. Our work is the first study of spatio-temporal distribution shifts in dynamic graphs, to the best of our knowledge.

Figures (8)

• Figure 1: The framework of our proposed method I-DIDA: 1. (Top) For a given dynamic graph with multiple timestamps, the disentangled dynamic graph attention networks first obtain summarizations of high-order invariant and variant patterns by disentangled spatio-temporal message passing. 2. (Middle) Then the spatio-temporal intervention mechanism creates multiple intervened distributions by sampling and reassembling variant patterns across space and time for each node. By utilizing the samples from the intervened distributions, the sample-level invariance loss is calculated to optimize the model so that it can focus on invariant patterns to make predictions. 3. (Bottom) Finally, the spatio-temporal environment inference module infers the environments by clustering the variant patterns, and an environment-level invariance loss is proposed to promote the invariance of the invariant patterns. In this way, the method can make predictions based on the invariant spatio-temporal patterns which have stable predictive abilities across distributions, and therefore handle the problem of distribution shifts on dynamic graphs. (Best viewed in color)
• Figure 2: Ablation studies on the environment inference, intervention mechanism and disentangled attention, where 'w/o I' removes the spatio-temporal environment inference module, 'w/o I&I' further removes the spatio-temporal intervention mechanism and 'w/o I&I&D' further removes disentangled attention. (Best viewed in color)
• Figure 3: Average neighbor degrees in the graph slice as time goes.
• Figure 4: Number of links in the graph slice as time goes.
• Figure 5: (a) Comparison of different intervention mechansim on COLLAB dataset, where I-DIDA-S only uses spatial intervention and I-DIDA-T only uses temporal intervention. (b) Comparison in terms of training time for each epoch on COLLAB dataset, where 'w/o I' means removing intervention mechanism in I-DIDA. (Best viewed in color)

Theorems & Definitions (1)

• Remark 1