Multi-times Monte Carlo Rendering for Inter-reflection Reconstruction

TL;DR

This work proposes Ref-MC2 that introduces the multi-time Monte Carlo sampling which comprehensively computes the environmental illumination and meanwhile considers the reflective light from object surfaces, and introduces a reflection-aware surface model to initialize the geometry and refine it during inverse rendering.

Abstract

Inverse rendering methods have achieved remarkable performance in reconstructing high-fidelity 3D objects with disentangled geometries, materials, and environmental light. However, they still face huge challenges in reflective surface reconstruction. Although recent methods model the light trace to learn specularity, the ignorance of indirect illumination makes it hard to handle inter-reflections among multiple smooth objects. In this work, we propose Ref-MC2 that introduces the multi-time Monte Carlo sampling which comprehensively computes the environmental illumination and meanwhile considers the reflective light from object surfaces. To address the computation challenge as the times of Monte Carlo sampling grow, we propose a specularity-adaptive sampling strategy, significantly reducing the computational complexity. Besides the computational resource, higher geometry accuracy is also required because geometric errors accumulate multiple times. Therefore, we further introduce a reflection-aware surface model to initialize the geometry and refine it during inverse rendering. We construct a challenging dataset containing scenes with multiple objects and inter-reflections. Experiments show that our method outperforms other inverse rendering methods on various object groups. We also show downstream applications, e.g., relighting and material editing, to illustrate the disentanglement ability of our method.

Figures (8)

• Figure 1: We perform Monte Carlo sampling at the viewpoint. When the sampling ray from the point is not blocked, it is the direct illumination from environmental lighting. When the sampling ray hits an object, we divide this indirect illumination from the object into diffuse light and specular light. We sample the diffuse light from a diffuse map that is optimized through self-supervision. For specular light, we only need to partially trace the rays in a small specular lobe along the reflective direction. The gradients are backward along the tracing path, and are passed to optimize $\boldsymbol{k}_d, \boldsymbol{k}_{orm}$, normals, and environment maps.
• Figure 2: Differences between previous SDF architectures and the architecture for inter-reflections.
• Figure 3: Qualitative comparison. The results of renderings, materials, and environment maps are presented. Note that, the material of Nefii contains only roughness without metalness. Our method achieves the best renderings with clear reflections, compared to other inverse rendering methods. Our method is also superior to others in the disentanglement of materials and environment maps.
• Figure 4: Ablation study on the depth of ray tracing, i.e., the times of Monte Carlo sampling. The results with depth=1 show fewer and darker reflections compared to the ground truth. $k_d$ maps also illustrate the limited capacity to disentangle the material from environmental light, for example, mistaking the diffuse color of the table as the color of the sky. In contrast, with depth=2 or 3, results show more realistic renderings and disentangled materials.
• Figure 5: Ablation study on geometric initialization. As shown, a better-quality geometry can significantly improve the material learning and also refine the rendering results.