Two-Stage Sketch-Based Smoke Illustration Generation using Stream Function

TL;DR

The paper addresses the challenge of creating expressive smoke illustrations without heavy physics-based tools by introducing a two-stage framework that maps a user sketch to a stream function and then to a velocity field via latent diffusion models. It employs Helmholtz–Hodge decomposition to derive a 2D stream function $\boldsymbol{\psi}$ from the velocity field, with a curl operation yielding the velocity field $\mathbf{U}$ used to drive a 2D incompressible smoke simulation, and uses streamlines as training priors to capture global flow dynamics. The main contributions are the integration of a stream function intermediate representation within a two-stage LDM, quantitative improvements in MSE over existing methods, and qualitative gains in preserving flow topology and detail plus an anime-style transfer interface. This work enhances accessibility and control in sketch-based smoke illustration by embedding physical flow priors into diffusion-based generation, enabling artists to achieve rotation-aware, continuous flows with simple sketches.

Abstract

In this paper, we propose a two-stage sketch-based smoke illustration generation framework using stream function and latent diffusion models (LDM). The user sketch is used to guide the generation of the stream function, which serves as the control condition for the velocity field generator. The generated velocity field can be used to guide the smoke simulation to align with the intended flow. We adopt streamlines to encode global flow dynamics as sketch guidance during training. The stream function constitutes the intermediate representation that captures continuous variation and rotational flow details absent from sketches.

Figures (3)

• Figure 1: The workflow of our proposed sketch-based smoke illustration framework using stream function.
• Figure 2: The qualitative evaluation result samples. The visualized velocity field is normalized into [0, 1] range.
• Figure 3: The anime style transferred smoke samples.