MLDGG: Meta-Learning for Domain Generalization on Graphs

TL;DR

This framework, MLDGG, endeavors to achieve adaptable generalization across diverse domains by integrating cross-multi-domain meta-learning with structure learning and semantic identification, and demonstrates that MLDGG surpasses baseline methods.

Abstract

Domain generalization on graphs aims to develop models with robust generalization capabilities, ensuring effective performance on the testing set despite disparities between testing and training distributions. However, existing methods often rely on static encoders directly applied to the target domain, constraining its flexible adaptability. In contrast to conventional methodologies, which concentrate on developing specific generalized models, our framework, MLDGG, endeavors to achieve adaptable generalization across diverse domains by integrating cross-multi-domain meta-learning with structure learning and semantic identification. Initially, it introduces a generalized structure learner to mitigate the adverse effects of task-unrelated edges, enhancing the comprehensiveness of representations learned by Graph Neural Networks (GNNs) while capturing shared structural information across domains. Subsequently, a representation learner is designed to disentangle domain-invariant semantic and domain-specific variation information in node embedding by leveraging causal reasoning for semantic identification, further enhancing generalization. In the context of meta-learning, meta-parameters for both learners are optimized to facilitate knowledge transfer and enable effective adaptation to graphs through fine-tuning within the target domains, where target graphs are inaccessible during training. Our empirical results demonstrate that MLDGG surpasses baseline methods, showcasing its effectiveness in three different distribution shift settings.

Figures (7)

• Figure 1: Visualizations of distribution shifts on graphs demonstrated using energy scores wu2023gnnsafe of nodes across different graphs, where each graph is sampled from a distinct domain characterized by variations of node features and topological structures simultaneously. The legend of all sub-figures follows the same naming format, i.e., "DataName - DomainName". (Left) Graphs are sampled from the same dataset Twitchrozemberczki2021multi. (Middle and Right) Graphs are sampled from different datasets, Twitch, Fb-100traud2012social, and WebKBpei2020geom.
• Figure 2: An overview of MLDGG. Each source graph is viewed as a task. For each task, the parameters $\{ \boldsymbol{\theta}'_t,\boldsymbol{\theta}'_g,\boldsymbol{\theta}'_r\}$ of the structure learner ($f_t$), GNN, and representation learner ($f_r$) are updated via $\mathcal{L}_{sup}$ during the inner update phase. Subsequently, the query losses $\mathcal{L}_{qry}$ across all tasks are aggregated to update the meta-parameters $\boldsymbol{\theta} =\{ \boldsymbol{\theta}_t, \boldsymbol{\theta}_r\}$ in the outer update phase. To generalize to graphs in the target domain, the learned meta-parameters of the structure learner and the representation learner are further fine-tuned for adaptation.
• Figure 3: Ablation study for MLDGG under three distinct cross-domain scenarios.
• Figure 4: Ablation study for MLDGG-ind under three distinct cross-domain scenarios.
• Figure 5: The demonstration of the effectiveness of representation learner on $3$ different setting S12T3, S1T2, and S1T1 using T-sne visualization. $\mathbf{r}$ denotes the output of GNN, $\mathbf{s}=E_s(\mathbf{r})$ denotes the semantic factor and $\mathbf{v}=E_v(\mathbf{r})$ denotes the variation factor. Different colors represent different labels.

Theorems & Definitions (7)

• Theorem 1: Upper bound: accuracy
• Theorem 2: Lower bound: accuracy
• proof
• Lemma 1
• Lemma 2
• proof
• Lemma 3