GCAL: Adapting Graph Models to Evolving Domain Shifts

TL;DR

GCAL tackles the challenge of unsupervised continual graph domain adaptation across evolving OOD graphs by coupling an adapt-with-memory-replay mechanism with a variational memory graph generator guided by an information bottleneck objective. The inner loop continuously adapts to new graphs using information-maximization while replaying memories to prevent forgetting, and the outer loop generates compact, informative memory graphs through a lower-bound optimization that includes gradient-based condensation and regularization terms. Key contributions include a theoretical IB-derived lower bound for memory generation, a three-loss memory-learning module, and a bi-level optimization framework that achieves superior adaptation and memory retention across diverse graph-domain shifts. Empirically, GCAL outperforms state-of-the-art baselines on regional and temporal graph shifts, demonstrating strong resilience to forgetting and practical potential for scalable continual graph learning.

Abstract

This paper addresses the challenge of graph domain adaptation on evolving, multiple out-of-distribution (OOD) graphs. Conventional graph domain adaptation methods are confined to single-step adaptation, making them ineffective in handling continuous domain shifts and prone to catastrophic forgetting. This paper introduces the Graph Continual Adaptive Learning (GCAL) method, designed to enhance model sustainability and adaptability across various graph domains. GCAL employs a bilevel optimization strategy. The "adapt" phase uses an information maximization approach to fine-tune the model with new graph domains while re-adapting past memories to mitigate forgetting. Concurrently, the "generate memory" phase, guided by a theoretical lower bound derived from information bottleneck theory, involves a variational memory graph generation module to condense original graphs into memories. Extensive experimental evaluations demonstrate that GCAL substantially outperforms existing methods in terms of adaptability and knowledge retention.

Figures (7)

• Figure 1: (a) The challenge of continual adaptation of graph models on evolving OOD graph sequences. (b) Empirical evaluations of the SOTA graph adaptation method across four OOD graph datasets in a continual adaptation setting.
• Figure 2: The GCAL framework involves several steps: Starting with the graph model $f(\Theta_{t-1})$, the current graph $G_t$, and the accumulated memory graph pool $\mathcal{G} = \{\widehat{G}_i\}_{i=1}^{t-1}$, GCAL first applies the Adaptation with Memory Replay method using loss $\mathcal{L}_{AMR}$ for model adaptation. Next, a Variational Memory Generator creates a new memory graph $\widehat{G}_t$ for $G_t$, which is refined using the memory graph learning loss $\mathcal{L}_{MGL}$, the regularization losses $\mathcal{L}_{Reg}$ to ensure stability and informativeness, and the generation loss $\mathcal{L}_{Gen}$ to enhance the memory graph's relevance to $G_t$. Finally, $\widehat{G}_t$ is added to $\mathcal{G}$ for future adaptation.
• Figure 3: Performance matrices of GCAL and CoTTA in different datasets.
• Figure 4: The model performance with different synthetic ratios.
• Figure 5: Visualized comparison of the original graphs (the first line) and generative graphs (the second line) of GCAL in the Twitch dataset.

Theorems & Definitions (3)

• Theorem 3.1
• Theorem 1.1
• proof