Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CoroNetGAN: Controlled Pruning of GANs via Hypernetworks

Aman Kumar, Khushboo Anand, Shubham Mandloi, Ashutosh Mishra, Avinash Thakur, Neeraj Kasera, Prathosh A P

TL;DR

This work proposes CoroNet-GAN for compressing GAN using the combined strength of differentiable pruning method via hypernetworks, which provides the advantage of performing controllable compression while training along with reducing training time by a substantial factor.

Abstract

Generative Adversarial Networks (GANs) have proven to exhibit remarkable performance and are widely used across many generative computer vision applications. However, the unprecedented demand for the deployment of GANs on resource-constrained edge devices still poses a challenge due to huge number of parameters involved in the generation process. This has led to focused attention on the area of compressing GANs. Most of the existing works use knowledge distillation with the overhead of teacher dependency. Moreover, there is no ability to control the degree of compression in these methods. Hence, we propose CoroNet-GAN for compressing GAN using the combined strength of differentiable pruning method via hypernetworks. The proposed method provides the advantage of performing controllable compression while training along with reducing training time by a substantial factor. Experiments have been done on various conditional GAN architectures (Pix2Pix and CycleGAN) to signify the effectiveness of our approach on multiple benchmark datasets such as Edges-to-Shoes, Horse-to-Zebra and Summer-to-Winter. The results obtained illustrate that our approach succeeds to outperform the baselines on Zebra-to-Horse and Summer-to-Winter achieving the best FID score of 32.3 and 72.3 respectively, yielding high-fidelity images across all the datasets. Additionally, our approach also outperforms the state-of-the-art methods in achieving better inference time on various smart-phone chipsets and data-types making it a feasible solution for deployment on edge devices.

CoroNetGAN: Controlled Pruning of GANs via Hypernetworks

TL;DR

This work proposes CoroNet-GAN for compressing GAN using the combined strength of differentiable pruning method via hypernetworks, which provides the advantage of performing controllable compression while training along with reducing training time by a substantial factor.

Abstract

Generative Adversarial Networks (GANs) have proven to exhibit remarkable performance and are widely used across many generative computer vision applications. However, the unprecedented demand for the deployment of GANs on resource-constrained edge devices still poses a challenge due to huge number of parameters involved in the generation process. This has led to focused attention on the area of compressing GANs. Most of the existing works use knowledge distillation with the overhead of teacher dependency. Moreover, there is no ability to control the degree of compression in these methods. Hence, we propose CoroNet-GAN for compressing GAN using the combined strength of differentiable pruning method via hypernetworks. The proposed method provides the advantage of performing controllable compression while training along with reducing training time by a substantial factor. Experiments have been done on various conditional GAN architectures (Pix2Pix and CycleGAN) to signify the effectiveness of our approach on multiple benchmark datasets such as Edges-to-Shoes, Horse-to-Zebra and Summer-to-Winter. The results obtained illustrate that our approach succeeds to outperform the baselines on Zebra-to-Horse and Summer-to-Winter achieving the best FID score of 32.3 and 72.3 respectively, yielding high-fidelity images across all the datasets. Additionally, our approach also outperforms the state-of-the-art methods in achieving better inference time on various smart-phone chipsets and data-types making it a feasible solution for deployment on edge devices.
Paper Structure (19 sections, 9 equations, 5 figures, 3 tables, 1 algorithm)

This paper contains 19 sections, 9 equations, 5 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Generative Adversarial Networks
  4. GAN Compression
  5. HyperNetworks
  6. Methodology
  7. HyperNetwork Design
  8. Vector Sparsity using Proximal Gradient
  9. Network Pruning
  10. Experiments
  11. Experiment Setting
  12. Models and Datasets
  13. Implementation Details
  14. Evaluation Setting
  15. Experimental Results
  16. ...and 4 more sections

Figures (5)

  • Figure 1: Illustration of the proposed algorithm designed for compressing GAN's using controllable differentiable pruning. A latent vector is attached to each of the convolution layer of the generator. The latent vector generates the weights for the generator via hypernetwork. Sparsification of the latent vector leads to pruning of the corresponding weights of the generator network. The proposed design allows the latent vector and its corresponding weight matrix to be covariant with each other. The generator generates visual results using the computed weight matrix through the hypernetwork (Best viewed when zoomed).
  • Figure 2: Graphical representation of training time(in minutes) and FID for Pix2Pix(left) on $\text{Edges}\rightarrow\text{Shoes}$ and CycleGAN(middle,right) on $\text{Horse}\rightarrow\text{Zebra}$ and $\text{Summer}\rightarrow\text{Winter}$ datasets respectively. From the graphs, it is evident that total training time for our proposed approach is significantly lesser compared to OMGD Ren2021OnlineMD. For CycleGAN on $\text{Summer}\rightarrow\text{Winter}$ dataset, our algorithm outperforms OMGD Ren2021OnlineMD on both training time and FID (Best viewed when zoomed).
  • Figure 3: Samples generated from our approach. First row contains translated images from $\text{Zebra}\rightarrow\text{Horse}$ dataset. The second row contains translated images from $\text{Horse}\rightarrow\text{Zebra}$ dataset (Best viewed when zoomed).
  • Figure 4: Qualitative comparison of CoroNetGAN with CycleGAN architecture on $\text{Summer}\rightarrow\text{Winter}$ dataset compared with original CycleGAN Zhu2017UnpairedIT, GAN Compression Li2020GANCE and OMGD Ren2021OnlineMD algorithms. Our approach generates visually realistic images and outperforms all the other algorithms on the FID metric (Best viewed when zoomed).
  • Figure 5: Qualitative comparison of CoroNetGAN with Pix2Pix architecture on $\text{Edges}\rightarrow\text{Shoes}$ dataset compared with original Pix2Pix Isola2017ImagetoImageTW, GAN Compression Li2020GANCE and OMGD Ren2021OnlineMD algorithms. Our approach generates visually plausible images compared to state-of-the-art methods (Best viewed when zoomed in).