Differentiable Iterated Function Systems

TL;DR

The paper tackles the inverse IFS problem by seeking a set of affine transformations whose attractor $A$ matches a target image, i.e., $A = H(A)$ with $H(S)=\cup_i F_i(S)$. It introduces a differentiable fractal rendering pipeline built in PyTorch that uses Signed Distance Functions (SDFs) and affine transforms to render IFS fractals and enables gradient-based optimization to minimize a loss against the target. Key contributions include an open-source code repository, analysis of gradient-descent pitfalls in fractal rasterization, best practice recommendations, and proposed directions for future experiments. The work demonstrates proof-of-concept feasibility on self-similar fractals like the Koch curve, while highlighting sensitivity to initialization and symmetry choices, pointing toward coarse-to-fine optimization and symmetry learning as promising avenues for broader applicability.

Abstract

This preliminary paper presents initial explorations in rendering Iterated Function System (IFS) fractals using a differentiable rendering pipeline. Differentiable rendering is a recent innovation at the intersection of computer graphics and machine learning. A fractal rendering pipeline composed of differentiable operations opens up many possibilities for generating fractals that meet particular criteria. In this paper I demonstrate this pipeline by generating IFS fractals with fixed points that resemble a given target image - a famous problem known as the \emph{inverse IFS problem}. The main contributions of this work are as follows: 1) I demonstrate (and make code available) this rendering pipeline; 2) I discuss some of the nuances and pitfalls in gradient-descent-based optimization over fractal structures; 3) I discuss best practices to address some of these pitfalls; and finally 4) I discuss directions for further experiments to validate the technique.

Figures (5)

• Figure 1: From left to right: First row: The first iteration of the Koch IFS fractal; the second iteration, showing its composition as several scaled and rotated copies of the first iteration. Second row: As in the second image, but without bounding boxes; the attractor of this process.
• Figure 2: Multiple examples of learning IFS attractors. In each row from top to bottom: the Koch curve; the Sierpiński carpet; an apple from the MPEG-7 shape dataset; a flower from the MPEG-7 shape dataset.
• Figure 3: Vector fields illustrating the gradient of error as $p_2$ is varied in the Koch curve construction, while all other points are held constant at their optimal positions: a) always pointing toward the optimal location (orange); b) the gradient of the fine-scale loss only; and c) the gradient of the multiscale loss. While both gradient vector fields are divergent, the multiscale loss is slightly more aligned with the always-optimal field.
• Figure 4: Examples of local optima encountered while optimizing a set of affine transformations to fit the Koch curve.
• Figure 5: Additional example fractals generated with the method outlined in this paper.