Inductive Graph Few-shot Class Incremental Learning

TL;DR

This work introduces inductive GFSCIL that continually learns novel classes with newly emerging nodes while maintaining performance on old classes without accessing previous data, and proposes an iterative prototype calibration to improve the separation of class prototypes.

Abstract

Node classification with Graph Neural Networks (GNN) under a fixed set of labels is well known in contrast to Graph Few-Shot Class Incremental Learning (GFSCIL), which involves learning a GNN classifier as graph nodes and classes growing over time sporadically. We introduce inductive GFSCIL that continually learns novel classes with newly emerging nodes while maintaining performance on old classes without accessing previous data. This addresses the practical concern of transductive GFSCIL, which requires storing the entire graph with historical data. Compared to the transductive GFSCIL, the inductive setting exacerbates catastrophic forgetting due to inaccessible previous data during incremental training, in addition to overfitting issue caused by label sparsity. Thus, we propose a novel method, called Topology-based class Augmentation and Prototype calibration (TAP). To be specific, it first creates a triple-branch multi-topology class augmentation method to enhance model generalization ability. As each incremental session receives a disjoint subgraph with nodes of novel classes, the multi-topology class augmentation method helps replicate such a setting in the base session to boost backbone versatility. In incremental learning, given the limited number of novel class samples, we propose an iterative prototype calibration to improve the separation of class prototypes. Furthermore, as backbone fine-tuning poses the feature distribution drift, prototypes of old classes start failing over time, we propose the prototype shift method for old classes to compensate for the drift. We showcase the proposed method on four datasets.

Figures (7)

• Figure 1: Transductive GFSCIL in Fig. \ref{['fig:transductive']}vs. our inductive GFSCIL in Fig. \ref{['fig:inductive']}. The classes are distinguished by distinct colors, with grey indicating unlabeled nodes for testing. Graphs that emerged at distinct sessions are enclosed by dashed lines. Notice that in transductive GFSCIL, the graph gets larger with each session, and links among session subgraphs are required (red color). In contrast, inductive GFSCIL does not store past subgraphs.
• Figure 2: The triple-branch multi-topology class augmentation used during the base session training.
• Figure 3: The training framework for model adaptation and prototype calibration in the incremental session.
• Figure 4: Performance comparison on the N-way 1-shot setting over different datasets.
• Figure 5: The visualization of prototype calibration for old and novel classes on Amazon_clothing, where the empty and colored stars indicate prototypes before and after calibration. The dotted lines denote trajectories of prototype movements.