Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AGALE: A Graph-Aware Continual Learning Evaluation Framework

Tianqi Zhao, Alan Hanjalic, Megha Khosla

TL;DR

AGALE tackles graph-structured continual learning with multi-label nodes by introducing two generalized incremental settings (Task-IL and Class-IL), novel data partitioning algorithms to generate coherent subgraphs, and a theoretical analysis of label homophily. It provides extensive cross-domain evaluation against CL, CGL, and DGL baselines, revealing how subgraph homophily and task structure influence forgetting and knowledge transfer. The framework enables fair, graph-aware evaluation of continual learning on graphs and offers practical guidance for designing methods that preserve past knowledge while handling evolving multi-label graphs. The public release of the framework facilitates future research into multi-label graph continual learning and selective forgetting strategies.

Abstract

In recent years, continual learning (CL) techniques have made significant progress in learning from streaming data while preserving knowledge across sequential tasks, particularly in the realm of euclidean data. To foster fair evaluation and recognize challenges in CL settings, several evaluation frameworks have been proposed, focusing mainly on the single- and multi-label classification task on euclidean data. However, these evaluation frameworks are not trivially applicable when the input data is graph-structured, as they do not consider the topological structure inherent in graphs. Existing continual graph learning (CGL) evaluation frameworks have predominantly focussed on single-label scenarios in the node classification (NC) task. This focus has overlooked the complexities of multi-label scenarios, where nodes may exhibit affiliations with multiple labels, simultaneously participating in multiple tasks. We develop a graph-aware evaluation (\agale) framework that accommodates both single-labeled and multi-labeled nodes, addressing the limitations of previous evaluation frameworks. In particular, we define new incremental settings and devise data partitioning algorithms tailored to CGL datasets. We perform extensive experiments comparing methods from the domains of continual learning, continual graph learning, and dynamic graph learning (DGL). We theoretically analyze \agale and provide new insights about the role of homophily in the performance of compared methods. We release our framework at https://github.com/Tianqi-py/AGALE.

AGALE: A Graph-Aware Continual Learning Evaluation Framework

TL;DR

AGALE tackles graph-structured continual learning with multi-label nodes by introducing two generalized incremental settings (Task-IL and Class-IL), novel data partitioning algorithms to generate coherent subgraphs, and a theoretical analysis of label homophily. It provides extensive cross-domain evaluation against CL, CGL, and DGL baselines, revealing how subgraph homophily and task structure influence forgetting and knowledge transfer. The framework enables fair, graph-aware evaluation of continual learning on graphs and offers practical guidance for designing methods that preserve past knowledge while handling evolving multi-label graphs. The public release of the framework facilitates future research into multi-label graph continual learning and selective forgetting strategies.

Abstract

In recent years, continual learning (CL) techniques have made significant progress in learning from streaming data while preserving knowledge across sequential tasks, particularly in the realm of euclidean data. To foster fair evaluation and recognize challenges in CL settings, several evaluation frameworks have been proposed, focusing mainly on the single- and multi-label classification task on euclidean data. However, these evaluation frameworks are not trivially applicable when the input data is graph-structured, as they do not consider the topological structure inherent in graphs. Existing continual graph learning (CGL) evaluation frameworks have predominantly focussed on single-label scenarios in the node classification (NC) task. This focus has overlooked the complexities of multi-label scenarios, where nodes may exhibit affiliations with multiple labels, simultaneously participating in multiple tasks. We develop a graph-aware evaluation (\agale) framework that accommodates both single-labeled and multi-labeled nodes, addressing the limitations of previous evaluation frameworks. In particular, we define new incremental settings and devise data partitioning algorithms tailored to CGL datasets. We perform extensive experiments comparing methods from the domains of continual learning, continual graph learning, and dynamic graph learning (DGL). We theoretically analyze \agale and provide new insights about the role of homophily in the performance of compared methods. We release our framework at https://github.com/Tianqi-py/AGALE.
Paper Structure (46 sections, 1 theorem, 11 equations, 15 figures, 9 tables, 2 algorithms)

This paper contains 46 sections, 1 theorem, 11 equations, 15 figures, 9 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Limitations of current continual learning evaluation frameworks
  3. Lack of graph-aware data partitioning strategies.
  4. Generation of train/val/test sets.
  5. Use of predefined class orders.
  6. Limitations on the number of tasks.
  7. Our Contributions
  8. Problem Formulation
  9. Differences to single-label scenario
  10. AGALE: our evaluation framework
  11. Two Generalized Incremental Settings for Continual Graph Learning
  12. Data Partitioning Algorithms
  13. Theoretical Analysis Of AGALE
  14. Comparison With Previous Evaluation Frameworks
  15. Related Work
  16. ...and 31 more sections

Key Result

Theorem 1

For any edge $e(i,j) \in \mathcal{E}$ and any subgraph at time $t$, $\mathcal{G}_t$ such that $e(i,j) \in \mathcal{E}_t$, $h^{e(i,j)}_{\mathcal{G}_t}\ge h^{e(i,j)}_{\mathcal{G}}$ when at least one of the nodes in $\{i,j\}$ is single-labeled. For the case when both nodes $i,j$ are multi-labeled, we o

Figures (15)

  • Figure 1: An example multi-label graph with colors indicating to the different node labels.
  • Figure 2: Subgraphs generated by grouping frequently co-occurring classes as a task.
  • Figure 3: The split of the nodes within one subgraph generated by the previous CGL framework.
  • Figure 4: An example of subgraphs obtained by applying different class orders for the static multi-label graph in Figure \ref{['fig:multi_label_graph']}.
  • Figure 5: Visualization of our proposed generalized evaluation CGL framework AGALE.
  • ...and 10 more figures

Theorems & Definitions (4)

  • Definition 1
  • Theorem 1
  • proof
  • Definition 2