Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Future-Proofing Class-Incremental Learning

Quentin Jodelet, Xin Liu, Yin Jun Phua, Tsuyoshi Murata

TL;DR

This paper tackles exemplar-free class-incremental learning by addressing the limited information available at the first incremental step. It introduces Future-Proofing Class-Incremental Learning (FPCIL), which jointly trains a frozen feature extractor on real initial data and synthetic future-class images generated by a pre-trained diffusion model, then discards future-class weights and proceeds with a standard incremental classifier update. Across CIFAR100 and ImageNet-Subset, FPCIL consistently improves average incremental accuracy, often outperforming real-data auxiliary pre-training and enabling notable gains especially when the initial step has few classes. The work demonstrates that synthetic future data can be more cost-effective and data-efficient than collecting real auxiliary datasets, while preserving or enhancing performance, with ablations highlighting the roles of future-class prediction accuracy, diffusion-diversity, backbone choice, and pre-training combinations.

Abstract

Exemplar-Free Class Incremental Learning is a highly challenging setting where replay memory is unavailable. Methods relying on frozen feature extractors have drawn attention recently in this setting due to their impressive performances and lower computational costs. However, those methods are highly dependent on the data used to train the feature extractor and may struggle when an insufficient amount of classes are available during the first incremental step. To overcome this limitation, we propose to use a pre-trained text-to-image diffusion model in order to generate synthetic images of future classes and use them to train the feature extractor. Experiments on the standard benchmarks CIFAR100 and ImageNet-Subset demonstrate that our proposed method can be used to improve state-of-the-art methods for exemplar-free class incremental learning, especially in the most difficult settings where the first incremental step only contains few classes. Moreover, we show that using synthetic samples of future classes achieves higher performance than using real data from different classes, paving the way for better and less costly pre-training methods for incremental learning.

Future-Proofing Class-Incremental Learning

TL;DR

This paper tackles exemplar-free class-incremental learning by addressing the limited information available at the first incremental step. It introduces Future-Proofing Class-Incremental Learning (FPCIL), which jointly trains a frozen feature extractor on real initial data and synthetic future-class images generated by a pre-trained diffusion model, then discards future-class weights and proceeds with a standard incremental classifier update. Across CIFAR100 and ImageNet-Subset, FPCIL consistently improves average incremental accuracy, often outperforming real-data auxiliary pre-training and enabling notable gains especially when the initial step has few classes. The work demonstrates that synthetic future data can be more cost-effective and data-efficient than collecting real auxiliary datasets, while preserving or enhancing performance, with ablations highlighting the roles of future-class prediction accuracy, diffusion-diversity, backbone choice, and pre-training combinations.

Abstract

Exemplar-Free Class Incremental Learning is a highly challenging setting where replay memory is unavailable. Methods relying on frozen feature extractors have drawn attention recently in this setting due to their impressive performances and lower computational costs. However, those methods are highly dependent on the data used to train the feature extractor and may struggle when an insufficient amount of classes are available during the first incremental step. To overcome this limitation, we propose to use a pre-trained text-to-image diffusion model in order to generate synthetic images of future classes and use them to train the feature extractor. Experiments on the standard benchmarks CIFAR100 and ImageNet-Subset demonstrate that our proposed method can be used to improve state-of-the-art methods for exemplar-free class incremental learning, especially in the most difficult settings where the first incremental step only contains few classes. Moreover, we show that using synthetic samples of future classes achieves higher performance than using real data from different classes, paving the way for better and less costly pre-training methods for incremental learning.
Paper Structure (25 sections, 2 figures, 12 tables, 1 algorithm)

This paper contains 25 sections, 2 figures, 12 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Class-Incremental Learning (CIL) and Exemplar-Free Class-Incremental Learning (EFCIL)
  4. Training on synthetic images
  5. Additional data for Incremental Learning
  6. Preparing for the future
  7. Proposed method
  8. Future-Proofing Class-Incremental Learning
  9. Generating future data
  10. Experiments
  11. Experimental setup
  12. Datasets
  13. Model training
  14. Synthetic generation
  15. Baselines
  16. ...and 10 more sections

Figures (2)

  • Figure 1: Diagram representing our proposed method Future-Proof Class Incremental Learning (FPCIL) which prepares the feature extractor for the future by jointly training it on an auxiliary dataset in addition to the current dataset during the initial step. The auxiliary dataset contains synthetic samples of future classes generated using a pre-trained text-to-image diffusion model. After the first incremental step, the feature extractor $\phi$ is frozen and the weights of the classifier $\tilde{w}$ corresponding to the future classes are removed. Afterward, in the following incremental step, only the classifier $\mathcal{W}$ is trained using dedicated method (e.g. FeTrIL, FeCAM).
  • Figure 2: Synthetic images of future classes for CIFAR100 generated using different diffusion models. Images are displayed before being resized for better appreciation. First row has been generated by Stable Diffusion 1.4 with a guidance scale of $2.0$, second row using Stable Diffusion 1.4 with a guidance scale of $7.5$, and last row using DALL-E2. From left to right: butterfly, can, house, lion, mountain, orange, otter, pine tree, rose, and streetcar.