Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Order-Robust Class Incremental Learning: Graph-Driven Dynamic Similarity Grouping

Guannan Lai, Yujie Li, Xiangkun Wang, Junbo Zhang, Tianrui Li, Xin Yang

TL;DR

Graph-Driven Dynamic Similarity Grouping (GDDSG), a novel method that employs graph coloring algorithms to dynamically partition classes into similarity-constrained groups, is proposed, proving that grouping classes with lower pairwise similarity during incremental phases significantly improves model robustness to order variations.

Abstract

Class Incremental Learning (CIL) aims to enable models to learn new classes sequentially while retaining knowledge of previous ones. Although current methods have alleviated catastrophic forgetting (CF), recent studies highlight that the performance of CIL models is highly sensitive to the order of class arrival, particularly when sequentially introduced classes exhibit high inter-class similarity. To address this critical yet understudied challenge of class order sensitivity, we first extend existing CIL frameworks through theoretical analysis, proving that grouping classes with lower pairwise similarity during incremental phases significantly improves model robustness to order variations. Building on this insight, we propose Graph-Driven Dynamic Similarity Grouping (GDDSG), a novel method that employs graph coloring algorithms to dynamically partition classes into similarity-constrained groups. Each group trains an isolated CIL sub-model and constructs meta-features for class group identification. Experimental results demonstrate that our method effectively addresses the issue of class order sensitivity while achieving optimal performance in both model accuracy and anti-forgetting capability. Our code is available at https://github.com/AIGNLAI/GDDSG.

Order-Robust Class Incremental Learning: Graph-Driven Dynamic Similarity Grouping

TL;DR

Graph-Driven Dynamic Similarity Grouping (GDDSG), a novel method that employs graph coloring algorithms to dynamically partition classes into similarity-constrained groups, is proposed, proving that grouping classes with lower pairwise similarity during incremental phases significantly improves model robustness to order variations.

Abstract

Class Incremental Learning (CIL) aims to enable models to learn new classes sequentially while retaining knowledge of previous ones. Although current methods have alleviated catastrophic forgetting (CF), recent studies highlight that the performance of CIL models is highly sensitive to the order of class arrival, particularly when sequentially introduced classes exhibit high inter-class similarity. To address this critical yet understudied challenge of class order sensitivity, we first extend existing CIL frameworks through theoretical analysis, proving that grouping classes with lower pairwise similarity during incremental phases significantly improves model robustness to order variations. Building on this insight, we propose Graph-Driven Dynamic Similarity Grouping (GDDSG), a novel method that employs graph coloring algorithms to dynamically partition classes into similarity-constrained groups. Each group trains an isolated CIL sub-model and constructs meta-features for class group identification. Experimental results demonstrate that our method effectively addresses the issue of class order sensitivity while achieving optimal performance in both model accuracy and anti-forgetting capability. Our code is available at https://github.com/AIGNLAI/GDDSG.

Paper Structure

This paper contains 23 sections, 2 theorems, 34 equations, 6 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Formulation and Theory Analysis
  4. Problem Formulation
  5. The Effect of Class Ordering in CIL
  6. The Proposed Method: GDDSG
  7. Class Grouping Based on Similarity
  8. Graph-Driven Class Grouping
  9. Overall Process
  10. Experiment
  11. Experimental Setup
  12. Experimental Results
  13. Conclusion & Limitation
  14. Notation
  15. Datesets, Implementations and Additional Experimental Results
  16. ...and 8 more sections

Key Result

Theorem 1

When $p \ge n + 2$, we must have: where the overparameterization ratio $r = 1 - \frac{n}{p}$ in this context quantifies the degree of overparameterization in a model, where $n$ represents the sample size, and $p$ denotes the number of model parameters muthukumar2020harmlesshastie2022surprises. The coefficients $c_{i,j} = (1 - r)(r^{

Figures (6)

  • Figure 1: The crucial challenges of CIL (illustration on CIFAR100 dataset). On the left subfigure (a), each model’s performance is shown under varying class orders, testing its robustness to class order sensitivity. On the right subfigure (b), the model’s performance is shown when classes within the same task are similar, evaluating its resilience to intra-task classes with high similarities.
  • Figure 2: Illustration of The Overall Framework. [best view in color]
  • Figure 3: Contour plot delineating the subthreshold region where $P_{\text{Satisfy Brooks}'} < 0.99$. The horizontal axis spans $p \in [0.9, 1.0]$, representing probability values, while the vertical axis specifies sample sizes $N \in [10, 40]$. In regions not displayed, the corresponding $P_{\text{Satisfy Brooks}'}$ values exceed 0.99.
  • Figure 4: Robustness of Different Methods to Class Order: MOPD (Blue) and AOPD (Orange) Indicators, with GDDSG Performing Well Across Four Datasets
  • Figure 5: Analysis of Class Group Counts: The Left Figure Shows Changes in Class Group Counts as the Number of Tasks Increases, and the Right Figure Shows Changes as Task Length Varies.
  • ...and 1 more figures

Theorems & Definitions (4)

  • Definition 1
  • Theorem 1
  • Corollary 1
  • Definition 2