Learning Image Fractals Using Chaotic Differentiable Point Splatting

TL;DR

This work tackles the fractal inverse problem by recovering fractal codes from a single image using a differentiable forward model that combines a parallel chaos-game fractal generator with differentiable point splatting. A hybrid optimization, alternating gradient-based updates with simulated annealing, navigates the highly non-convex landscape to produce high-quality fractal codes that can synthesize infinite-resolution detail. The approach achieves state-of-the-art inversion results against diverse baselines and demonstrates robust zoom-ins, real-image inversions, and ablations validating its key components. The results suggest fractals can serve as scalable, optimizable graphics primitives with potential for extensions to 3D and appearance-aware rendering.

Abstract

Fractal geometry, defined by self-similar patterns across scales, is crucial for understanding natural structures. This work addresses the fractal inverse problem, which involves extracting fractal codes from images to explain these patterns and synthesize them at arbitrary finer scales. We introduce a novel algorithm that optimizes Iterated Function System parameters using a custom fractal generator combined with differentiable point splatting. By integrating both stochastic and gradient-based optimization techniques, our approach effectively navigates the complex energy landscapes typical of fractal inversion, ensuring robust performance and the ability to escape local minima. We demonstrate the method's effectiveness through comparisons with various fractal inversion techniques, highlighting its ability to recover high-quality fractal codes and perform extensive zoom-ins to reveal intricate patterns from just a single image.

Figures (7)

• Figure 1: A fern exhibiting self-similarities.
• Figure 2: An IFS fractal generated using the chaos game (Eq. \ref{['eq:chaos_game']}) with varying numbers of points.
• Figure 3: Overview of our model. The fractal point generator uses functions in $\TextOrMath{$F$\xspace}{\mathcal{F}}$ and employs a parallel stochastic scheme to efficiently and robustly execute the chaos game (Eq. \ref{['eq:chaos_game']}). The generated points are then rendered into an image using differentiable splatting.
• Figure 4: Qualitative fractal inversion results for various methods (rows). For each instance (columns), the full fractal is shown on the left, with a 64x zoomed-in view on the right. The bottom row presents the input image ${I_\textrm{ref}}$I_ref used for all methods (left) next to a zoomed-in view of the ground-truth fractal (right). Note that the zoomed-in views are consistent across rows. Images marked with a $\blacktriangle$ denote views that do not contain the full point count (see Sec. \ref{['sec:Evaluation']}). More visuals can be found in our supplemental materials. For continuous zoom-in visuals, please refer to our supplemental video.
• Figure 5: Inversion results for real images. Our representation enables infinite zoom-ins, continuously generating self-similar details at any scale while capturing intricate structural patterns. In contrast, natural structures exhibit self-similarity only within a limited scale range.