Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Near-Light Color Photometric Stereo for mono-Chromaticity non-lambertian surface

Zonglin Li, Jieji Ren, Shuangfan Zhou, Heng Guo, Jinnuo Zhang, Jiang Zhou, Boxin Shi, Zhanyu Ma, Guoying Gu

TL;DR

This work tackles single-shot color photometric stereo under near-field, non-Lambertian illumination by leveraging neural implicit representations for depth and BRDF under mono-chromaticity. It introduces a neural surface model to predict depth per pixel and derive normals from depth gradients, and a neural BRDF model with independent RGB branches guided by the Rusinkiewicz parameterization, all trained end-to-end with a differentiable near-light rendering equation $\hat{I}_{u,v} = \mathbf{r} \cdot \frac{\varphi}{\|\mathbf{l}-\mathbf{x}\|^2} \max(\mathbf{q}^\top \mathbf{n}, 0)$. The approach turns the color photometric stereo ill-posed problem into a well-posed one by exploiting mono-chromaticity and shared BRDF structure, enabling accurate normal, depth, and mesh recovery from a single snapshot. It is validated on synthetic data and in real-world scenarios using a compact visuo-tactile sensor (NearLightTactile), with a Crosstalk Correction Module improving photometric consistency in compact hardware. The results demonstrate robust and high-fidelity 3D reconstruction under challenging near-light and non-Lambertian conditions, highlighting practical implications for tactile sensing and in-situ Shape-from-Shading tasks.

Abstract

Color photometric stereo enables single-shot surface reconstruction, extending conventional photometric stereo that requires multiple images of a static scene under varying illumination to dynamic scenarios. However, most existing approaches assume ideal distant lighting and Lambertian reflectance, leaving more practical near-light conditions and non-Lambertian surfaces underexplored. To overcome this limitation, we propose a framework that leverages neural implicit representations for depth and BRDF modeling under the assumption of mono-chromaticity (uniform chromaticity and homogeneous material), which alleviates the inherent ill-posedness of color photometric stereo and allows for detailed surface recovery from just one image. Furthermore, we design a compact optical tactile sensor to validate our approach. Experiments on both synthetic and real-world datasets demonstrate that our method achieves accurate and robust surface reconstruction.

Near-Light Color Photometric Stereo for mono-Chromaticity non-lambertian surface

TL;DR

This work tackles single-shot color photometric stereo under near-field, non-Lambertian illumination by leveraging neural implicit representations for depth and BRDF under mono-chromaticity. It introduces a neural surface model to predict depth per pixel and derive normals from depth gradients, and a neural BRDF model with independent RGB branches guided by the Rusinkiewicz parameterization, all trained end-to-end with a differentiable near-light rendering equation . The approach turns the color photometric stereo ill-posed problem into a well-posed one by exploiting mono-chromaticity and shared BRDF structure, enabling accurate normal, depth, and mesh recovery from a single snapshot. It is validated on synthetic data and in real-world scenarios using a compact visuo-tactile sensor (NearLightTactile), with a Crosstalk Correction Module improving photometric consistency in compact hardware. The results demonstrate robust and high-fidelity 3D reconstruction under challenging near-light and non-Lambertian conditions, highlighting practical implications for tactile sensing and in-situ Shape-from-Shading tasks.

Abstract

Color photometric stereo enables single-shot surface reconstruction, extending conventional photometric stereo that requires multiple images of a static scene under varying illumination to dynamic scenarios. However, most existing approaches assume ideal distant lighting and Lambertian reflectance, leaving more practical near-light conditions and non-Lambertian surfaces underexplored. To overcome this limitation, we propose a framework that leverages neural implicit representations for depth and BRDF modeling under the assumption of mono-chromaticity (uniform chromaticity and homogeneous material), which alleviates the inherent ill-posedness of color photometric stereo and allows for detailed surface recovery from just one image. Furthermore, we design a compact optical tactile sensor to validate our approach. Experiments on both synthetic and real-world datasets demonstrate that our method achieves accurate and robust surface reconstruction.
Paper Structure (13 sections, 5 equations, 7 figures, 1 table)

This paper contains 13 sections, 5 equations, 7 figures, 1 table.

Table of Contents

  1. Introduction
  2. Method
  3. Neural Surface Model
  4. Neural BRDF Model
  5. Differentiable Rendering and Optimization
  6. Experiments
  7. Experimental Setup
  8. Experiment Results and Analysis
  9. Ablation Study
  10. Real-World Validation
  11. Robustness against Crosstalk
  12. Qualitative Results on Real-World Data
  13. Conclusion

Figures (7)

  • Figure 1: Our method takes a single captured RGB image and the near-light source position as input, and outputs accurate surface normal, depth map and mesh.
  • Figure 2: Our method has a neural surface model and a neural BRDF model. The neural surface model estimates depth and normal from image coordinates, the BRDF model predicts reflectance using the angle between surface normal, light and view directions. The system is optimized via photometric consistency loss between the rendered and captured images.
  • Figure 3: Synthetic dataset covering varying geometric details and reflection types.
  • Figure 4: Quantitative comparison of surface normal.
  • Figure 5: Ablation Study of our method.
  • ...and 2 more figures