Learnable Fractal Flames

TL;DR

This work presents a differentiable fractal rendering pipeline that learns latent fractal flame parameters from image supervision, extending prior differentiable IFS methods to color, non-linear generator functions, and multi-fractal compositions. Implemented in Taichi, the four-component pipeline (sampler, splatter, painter, and compositor) enables end-to-end gradient-based learning to fit reference images into colorful fractal flames. The experiments demonstrate learning with non-linear variations, exploring optimization dynamics, and composing multiple fractals to reproduce painting-like structures, while also discussing limitations such as training instability and lack of stochastic generator support. Collectively, the approach offers artists a fast, intuitive tool for generating rich fractal artwork from references and points toward interactive applications in existing fractal art software.

Abstract

This work presents a differentiable rendering approach that allows latent fractal flame parameters to be learned from image supervision using gradient descent optimization. The approach extends the state-of-the-art in differentiable iterated function system fractal rendering through support for color images, non-linear generator functions, and multi-fractal compositions. With this approach, artists can use reference images to quickly and intuitively control the creation of fractals. We describe the approach and conduct a series of experiments exploring its use, culminating in the creation of complex and colorful fractal artwork based on famous paintings.

Figures (7)

• Figure 1: The Barnsley fern is an example of a linear IFS fractal. When the parameters of the generator functions are perturbed (left vs. right) the fractal structure is altered. In general, it is difficult to know which fractal parameters to modify, and how, in order to achieve a particular effect.
• Figure 2: A flow diagram of our differentiable fractal rendering pipeline. First, a sampler uses a set of generator functions to produce arrays of sample positions and quality vectors. Second, a splatter splats each sample quality vector onto an image buffer at the corresponding sample position. Third, a painter maps the vector at each splat buffer pixel to an RGBA value. Finally, the compositor composites RGBA buffers from one or more fractals over a background color to produce the final image. The pipeline is end-to-end differentiable.
• Figure 3: Fractal flames learned from a simple reference image. Top row (left to right): the reference image, a linear variation, a spherical variation, a hankerchief variation. Bottom row (left to right): an exponential variation, a disk variation, a heart variation, a power variation.
• Figure 4: Top row: two fractal flames with 4 generator functions. Bottom row: two fractal flames with 16 generator functions. Fractals with fewer generator functions generally have simpler geometry and color palettes. All fractals are linear and were trained on the reference image shown in Figure \ref{['fig:variations']}.
• Figure 5: Four fractal flames with 8 generator functions that were trained using different initial parameter values. When the initial parameters are different, the optimization process will often find a different local minimum, rather that a common global minimum. All fractals are linear and were trained on the reference image shown in Figure \ref{['fig:variations']}.