Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CFTS-GAN: Continual Few-Shot Teacher Student for Generative Adversarial Networks

Munsif Ali, Leonardo Rossi, Massimo Bertozzi

TL;DR

A Continual Few-shot Teacher-Student technique for the generative adversarial network (CFTS-GAN) that considers both challenges together and demonstrates more diverse image synthesis and produce qualitative samples comparatively good to very stronger state-of-the-art models.

Abstract

Few-shot and continual learning face two well-known challenges in GANs: overfitting and catastrophic forgetting. Learning new tasks results in catastrophic forgetting in deep learning models. In the case of a few-shot setting, the model learns from a very limited number of samples (e.g. 10 samples), which can lead to overfitting and mode collapse. So, this paper proposes a Continual Few-shot Teacher-Student technique for the generative adversarial network (CFTS-GAN) that considers both challenges together. Our CFTS-GAN uses an adapter module as a student to learn a new task without affecting the previous knowledge. To make the student model efficient in learning new tasks, the knowledge from a teacher model is distilled to the student. In addition, the Cross-Domain Correspondence (CDC) loss is used by both teacher and student to promote diversity and to avoid mode collapse. Moreover, an effective strategy of freezing the discriminator is also utilized for enhancing performance. Qualitative and quantitative results demonstrate more diverse image synthesis and produce qualitative samples comparatively good to very stronger state-of-the-art models.

CFTS-GAN: Continual Few-Shot Teacher Student for Generative Adversarial Networks

TL;DR

A Continual Few-shot Teacher-Student technique for the generative adversarial network (CFTS-GAN) that considers both challenges together and demonstrates more diverse image synthesis and produce qualitative samples comparatively good to very stronger state-of-the-art models.

Abstract

Few-shot and continual learning face two well-known challenges in GANs: overfitting and catastrophic forgetting. Learning new tasks results in catastrophic forgetting in deep learning models. In the case of a few-shot setting, the model learns from a very limited number of samples (e.g. 10 samples), which can lead to overfitting and mode collapse. So, this paper proposes a Continual Few-shot Teacher-Student technique for the generative adversarial network (CFTS-GAN) that considers both challenges together. Our CFTS-GAN uses an adapter module as a student to learn a new task without affecting the previous knowledge. To make the student model efficient in learning new tasks, the knowledge from a teacher model is distilled to the student. In addition, the Cross-Domain Correspondence (CDC) loss is used by both teacher and student to promote diversity and to avoid mode collapse. Moreover, an effective strategy of freezing the discriminator is also utilized for enhancing performance. Qualitative and quantitative results demonstrate more diverse image synthesis and produce qualitative samples comparatively good to very stronger state-of-the-art models.

Paper Structure

This paper contains 11 sections, 7 equations, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Method
  4. Continual Few-Shot GAN
  5. Cross Domain Consistency Loss
  6. Teacher Student Model
  7. Experiments
  8. Qualitative Results
  9. Quantitative Results
  10. Ablation Study
  11. Conclusion

Figures (2)

  • Figure 1: The teacher-student model for continual learning GAN. The dashed lines show the frozen model. a) shows the teacher model training. b) demonstrates training of the student model.
  • Figure 2: Qualitative results: Generated samples. In each group, the first column is for sketches (Task 1), and the second, third, fourth, and fifth are for females (Task 2), sunglasses (Task 3), males (Task 4), and babies (Task 5).