DIAMOND-SSS: Diffusion-Augmented Multi-View Optimization for Data-efficient SubSurface Scattering

TL;DR

DIAMOND-SSS tackles the challenge of data-efficient translucent reconstruction by combining diffusion-based data augmentation with illumination-invariant geometric priors. By fine-tuning two geometry-conditioned diffusion models for novel-view synthesis and relighting on a small OLAT subset, the method generates photorealistic, geometrically consistent augmentations that greatly suppress the need for dense real data. When integrated with Subsurface Scattering Gaussian Splatting and reinforced by multi-view silhouette and depth consistency losses, the approach achieves high-fidelity relightable reconstructions under extreme sparsity, reducing real captures by up to 90% and replacing up to 95% of missing views or illuminations. The results demonstrate state-of-the-art quality in relightable Gaussian rendering for translucent materials and enable scalable data collection in unconstrained settings, with practical implications for realistic rendering of wax, jade, marble, and skin.

Abstract

Subsurface scattering (SSS) gives translucent materials -- such as wax, jade, marble, and skin -- their characteristic soft shadows, color bleeding, and diffuse glow. Modeling these effects in neural rendering remains challenging due to complex light transport and the need for densely captured multi-view, multi-light datasets (often more than 100 views and 112 OLATs). We present DIAMOND-SSS, a data-efficient framework for high-fidelity translucent reconstruction from extremely sparse supervision -- even as few as ten images. We fine-tune diffusion models for novel-view synthesis and relighting, conditioned on estimated geometry and trained on less than 7 percent of the dataset, producing photorealistic augmentations that can replace up to 95 percent of missing captures. To stabilize reconstruction under sparse or synthetic supervision, we introduce illumination-independent geometric priors: a multi-view silhouette consistency loss and a multi-view depth consistency loss. Across all sparsity regimes, DIAMOND-SSS achieves state-of-the-art quality in relightable Gaussian rendering, reducing real capture requirements by up to 90 percent compared to SSS-3DGS.

Figures (21)

• Figure 1: We propose a data-efficient framework for re-lightable 3D with subsurface scattering. Unlike prior work dihlmann2024subsurfacescattering3dgaussian requiring many viewpoints under multiple illuminations, our method uses diffusion models to synthesize novel views and lighting from sparse observations. The baseline dihlmann2024subsurfacescattering3dgaussian massively overfits in the second row, for the only given light condition.
• Figure 2: Overview of our data augmentation pipeline. From a few posed input images, we generate additional viewpoints using a novel-view diffusion model (green) and synthetic OLAT variants using a relighting model (yellow). The combined real and synthetic data are used to train SSS-3DGS dihlmann2024subsurfacescattering3dgaussian with an MLP for subsurface scattering, stabilized by our geometric consistency losses (\ref{['subsec:geometric_consistency']}).
• Figure 3: Visualization of the proposed geometric consistency losses. Silhouette consistency enforces cross-view contour alignment, while depth consistency stabilizes global geometry under varying lighting.
• Figure 4: Qualitative comparison of reconstructed translucent appearance under different supervision conditions, the first row shows the setting: all views, all lights.
• Figure 5: Parallel coordinate plot of the hyperparameter search. Each line corresponds to one configuration; lighter colors indicate higher values of the aggregated objective $\mathcal{L}_\text{total}$, as defined in Eq. \ref{['eq:avg_loss']}.