Cyclic image generation using chaotic dynamics

TL;DR

The results suggest that chaotic dynamics in the image space defined by the deep generative model contribute to the diversity of the generated images, constituting a novel approach for multi-class image generation.

Abstract

Successive image generation using cyclic transformations is demonstrated by extending the CycleGAN model to transform images among three different categories. Repeated application of the trained generators produces sequences of images that transition among the different categories. The generated image sequences occupy a more limited region of the image space compared with the original training dataset. Quantitative evaluation using precision and recall metrics indicates that the generated images have high quality but reduced diversity relative to the training dataset. Such successive generation processes are characterized as chaotic dynamics in terms of dynamical system theory. Positive Lyapunov exponents estimated from the generated trajectories confirm the presence of chaotic dynamics, with the Lyapunov dimension of the attractor found to be comparable to the intrinsic dimension of the training data manifold. The results suggest that chaotic dynamics in the image space defined by the deep generative model contribute to the diversity of the generated images, constituting a novel approach for multi-class image generation. This model can be interpreted as an extension of classical associative memory to perform hetero-association among image categories.

Figures (9)

• Figure 1: Transformations learned by model.
• Figure 2: Example of generated image sequences for MNIST. The leftmost image in each row is the initial image, and the subsequent images are generated by iteratively applying the generator $G$ to the previous image.
• Figure 3: Example of generated image sequences for Fashion-MNIST. The leftmost image in each row is the initial image, and the subsequent images are generated by iteratively applying the generator $G$ to the previous image.
• Figure 4: Visualization of distribution of training data and generated data by UMAP. Left: results for MNIST dataset. The green, pink, and cyan points represent 0, 1, and 2 image data in the training dataset, respectively, and the purple points represent the generated images. Right: results for Fashion-MNIST dataset. The green, pink, and cyan points represent T-shirt/top, sneaker, and bag image data in the training dataset, respectively, and the purple points represent the generated images.
• Figure 5: UMAP visualization of transitions of a set of states starting from a category $X$ image for MNIST dataset. The step $n$ indicates the number of transformations applied to the initial image. Green, pink, and cyan points represent image data for digits 0, 1, and 2 in the training dataset, respectively, and purple points represent the generated images. The area where the transitioned points are present shrinks over time relative to the area where the training data exist.