Monte Carlo Rendering to Diffusion Curves with Differential BEM

TL;DR

This work presents a method for generating vector graphics, in the form of diffusion curves, directly from noisy samples produced by a Monte Carlo renderer, using a novel differential boundary element method (BEM) framework that reconstructs colors from diffusion curve handles and computes gradients with respect to their parameters.

Abstract

We present a method for generating vector graphics, in the form of diffusion curves, directly from noisy samples produced by a Monte Carlo renderer. While generating raster images from 3D geometry via Monte Carlo raytracing is commonplace, there is no corresponding practical approach for robustly and directly extracting editable vector images with shading information from 3D geometry. To fill this gap, we formulate the problem as a stochastic optimization problem over the space of geometries and colors of diffusion curve handles, and solve it with the Levenberg-Marquardt algorithm. At the core of our method is a novel differential boundary element method (BEM) framework that reconstructs colors from diffusion curve handles and computes gradients with respect to their parameters, requiring the expensive matrix factorization only once at the beginning of the optimization. Unlike triangulation-based techniques that require a clean domain decomposition, our method is robust to geometrically challenging scenarios, such as intersecting diffusion curves, and to color noise in the target image, enabling the direct use of noisy Monte Carlo samples without requiring a converged, error-free input image. We demonstrate the robustness and broad applicability of our approach across several test cases. Finally, we highlight several open questions raised by our work, which spans both theory and applications.

Figures (9)

• Figure 1: Progressive optimization. Our method quickly captures the overall pattern of the image in a few minutes, and progressively refines the handles.
• Figure 2: Handle positions. Compared to the baseline, disabling either the handle length regularization or the endpoint snapping regularization results in handles that become arbitrarily long or are distributed without forming coherent structures, despite achieving a slight improvement in reconstruction error. When the handle position update is disabled, the output retains artificial patterns introduced by the initial handle configuration, and the reconstruction error is higher.
• Figure 3: Comparison of optimization methods. We compare Levenberg–Marquardt optimization with AdamW with three learning rates, scaled by powers of ten. Learning rates below 0.001 or above 0.1 result in slow convergence, while 0.01 performs best among the three. Nevertheless, even with this choice, AdamW converges more slowly than Levenberg–Marquardt.
• Figure 4: Discontinuity smoothing. We use a relatively large smoothing parameter of $\varepsilon = 10^{-2}$ (1% of the image width) in all experiments. Reducing $\varepsilon$ to $10^{-4}$ increases the quantitative reconstruction error and introduces perceptual artifacts in the form of sharp color discontinuities at undesired locations.
• Figure 5: Sparsity pruning. Results with different sparsity regularization constants $\lambda_\mathbf{w}$. All optimizations start with $500$ handles, and we report the final number of handles and RMSE after 100 steps. Larger values of $\lambda_\mathbf{w}$ produce sparser handle selections but also lead to higher errors, both quantitatively and qualitatively, manifesting as sharp color discontinuities and overly blurred regions.