Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Controlling the image generation process with parametric activation functions

Ilia Pavlov

TL;DR

The paper addresses the scarcity of interpretable tools for controlling image-generative models by enabling users to replace static activations with parametric variants and adjust their parameters through a GUI. It demonstrates the approach on StyleGAN2 and BigGAN trained on FFHQ and ImageNet, respectively, and shows how mapping and early generator layers respond to activation changes. The method introduces several activations, including SinLU, ReLUN, ShiLU, and a polynomial form, controlled by parameters $a$, $b$, $n$, and $w_i$, respectively. The work provides a practical, interactive framework for exploring internal mechanisms of image generation with educational value and acknowledges the need for a user study and guided control improvements.

Abstract

As image generative models continue to increase not only in their fidelity but also in their ubiquity the development of tools that leverage direct interaction with their internal mechanisms in an interpretable way has received little attention In this work we introduce a system that allows users to develop a better understanding of the model through interaction and experimentation By giving users the ability to replace activation functions of a generative network with parametric ones and a way to set the parameters of these functions we introduce an alternative approach to control the networks output We demonstrate the use of our method on StyleGAN2 and BigGAN networks trained on FFHQ and ImageNet respectively.

Controlling the image generation process with parametric activation functions

TL;DR

The paper addresses the scarcity of interpretable tools for controlling image-generative models by enabling users to replace static activations with parametric variants and adjust their parameters through a GUI. It demonstrates the approach on StyleGAN2 and BigGAN trained on FFHQ and ImageNet, respectively, and shows how mapping and early generator layers respond to activation changes. The method introduces several activations, including SinLU, ReLUN, ShiLU, and a polynomial form, controlled by parameters , , , and , respectively. The work provides a practical, interactive framework for exploring internal mechanisms of image generation with educational value and acknowledges the need for a user study and guided control improvements.

Abstract

As image generative models continue to increase not only in their fidelity but also in their ubiquity the development of tools that leverage direct interaction with their internal mechanisms in an interpretable way has received little attention In this work we introduce a system that allows users to develop a better understanding of the model through interaction and experimentation By giving users the ability to replace activation functions of a generative network with parametric ones and a way to set the parameters of these functions we introduce an alternative approach to control the networks output We demonstrate the use of our method on StyleGAN2 and BigGAN networks trained on FFHQ and ImageNet respectively.
Paper Structure (6 sections, 4 equations, 5 figures)

This paper contains 6 sections, 4 equations, 5 figures.

Table of Contents

  1. Introduction
  2. Background
  3. Control Method
  4. Activation functions
  5. Experiments
  6. Conclusion

Figures (5)

  • Figure 1: A Graphical User Interface was developed for our control method. The left side is responsible for picking neural layers, the center side is responsible for selecting activation functions and adjusting their parameters if applicable, and the right side allows for real-time result visualization.
  • Figure 2: Examples of using parametric activation functions in the mapping network of the StyleGAN2. The left-most images in output rows (faces) are base outputs. Each image has one more base activation function replaced with a parametric one, compared with an image to its' left. Bottom rows show plots of parametric functions, which were used to produce the images above.
  • Figure 3: Examples of using parametric activation functions in both mapping and generator networks of the StyleGAN2. The left-most images in output rows (faces) are base outputs. Each image has one more base activation function replaced with a parametric one, compared with an image to its' left. Bottom rows show plots of parametric functions, which were used to produce the images above.
  • Figure 4: Output of a BigGAN model. The left-most image is an original, the rest were generated with SinLU or ReLUN applied to the second layer of the network. The parameter values were randomly chosen.
  • Figure 5: Output of a BigGAN model with 3rd order polynomial activation functions. The functions' parameters were kept within $\left[0.8, 1.2\right]$ interval. The left-most images are originals, the right-most are generated with 4 modified functions, and the rest were generated with only one modified function.