Stable Diffusion with Continuous-time Neural Network

TL;DR

This paper exploration and demonstration of the potential of celllular neural networks in image generation will demonstrate their superiority in performance, showcasing their adeptness in producing higher quality images and achieving quicker training times in comparison to their discrete-time counterparts on the commonly cited MNIST dataset.

Abstract

Stable diffusion models have ushered in a new era of advancements in image generation, currently reigning as the state-of-the-art approach, exhibiting unparalleled performance. The process of diffusion, accompanied by denoising through iterative convolutional or transformer network steps, stands at the core of their implementation. Neural networks operating in continuous time naturally embrace the concept of diffusion, this way they could enable more accurate and energy efficient implementation. Within the confines of this paper, my focus delves into an exploration and demonstration of the potential of celllular neural networks in image generation. I will demonstrate their superiority in performance, showcasing their adeptness in producing higher quality images and achieving quicker training times in comparison to their discrete-time counterparts on the commonly cited MNIST dataset.

Figures (4)

• Figure 1: Depiction of the diffusion which generates random Gaussian noise from an input image. The approximated inverse of this process can be used for image generation which was demonstrated in rombach2022high.
• Figure 2: Typical architecture of a Latent Diffusion Model where the inverse of the diffusion step - which is also a diffusion process- is approximated by a series of discrete operations (self-attentions or convolutions). The original image was taken from rombach2022high.
• Figure 3: Illustration of the circuit of a multi-channel M-CellNN cell $C_{i,j,k}$ The input (output) stage lies on the leftmost (rightmost) side of the circuit.
• Figure 4: These images offer qualitative outcomes, presenting samples for comparative assessment. The images in the top row are results from models trained on the MNIST, and in the bottom row on the CIFAR-10 datasets. Images in the left column showcase samples produced by conventional stable diffusion models incorporating convolutional blocks. In the middle samples generated with CellNN and in the right column samples generated by the TaOx M-CellNN are displayed.