LIPIDS: Learning-based Illumination Planning In Discretized (Light) Space for Photometric Stereo

TL;DR

This paper addresses the problem of selecting an optimal, minimal set of lighting directions for photometric stereo without per-object online optimization. It introduces LIPIDS, a framework that discretizes the light space into bins and learns a universal $M$-light configuration via a Light Sampling Network (LSNet) trained with a Normal Regression Network (NRNet). The approach yields competitive or superior normal estimation across synthetic and real datasets, and across multiple PS backbones, while offering faster inference than online methods. By leveraging offline, distribution-agnostic lighting planning, LIPIDS provides a practical path to robust, scalable photometric stereo in varied illumination settings.

Abstract

Photometric stereo is a powerful method for obtaining per-pixel surface normals from differently illuminated images of an object. While several methods address photometric stereo with different image (or light) counts ranging from one to two to a hundred, very few focus on learning optimal lighting configuration. Finding an optimal configuration is challenging due to the vast number of possible lighting directions. Moreover, exhaustively sampling all possibilities is impractical due to time and resource constraints. Photometric stereo methods have demonstrated promising performance on existing datasets, which feature limited light directions sparsely sampled from the light space. Therefore, can we optimally utilize these datasets for illumination planning? In this work, we introduce LIPIDS - Learning-based Illumination Planning In Discretized light Space to achieve minimal and optimal lighting configurations for photometric stereo under arbitrary light distribution. We propose a Light Sampling Network (LSNet) that optimizes lighting direction for a fixed number of lights by minimizing the normal loss through a normal regression network. The learned light configurations can directly estimate surface normals during inference, even using an off-the-shelf photometric stereo method. Extensive qualitative and quantitative analyses on synthetic and real-world datasets show that photometric stereo under learned lighting configurations through LIPIDS either surpasses or is nearly comparable to existing illumination planning methods across different photometric stereo backbones.

Figures (11)

• Figure 1: Schematic diagram of the proposed method for illumination planning in photometric stereo. (a) Given a set of images under an arbitrary lighting distribution, we first discretize the light space into $K$ bins spanning the upper hemisphere, (b) followed by light assignment to an appropriate bin. For a desired $M < K$, we use Light Sampling Network (LSNet) to select an optimal $M$-light configuration such that (c) the normals estimated using this set of $M$ lights have the least mean angular error.
• Figure 2: Architecture of the LIPIDS framework with Light Sampling Network (LSNet) and Normal Regression Network (NRNet)
• Figure 3: Top: Different lighting distributions across the publicly available photometric stereo datasets. Bottom: Variation of MAE with different numbers of light bins evaluated over the DiLiGenT datasetshi2016benchmark. Note that for all values of $K$, we sample more along azimuth than elevation.
• Figure 4: Evolution of the optimal lighting configuration while training LSNet + NRNet over ten epochs. Each row represents the values in one column of $\widehat{W}$ matrix. The final converged light position represents the associated light bin in the discretized light space.
• Figure 5: The optimal $M$-light distribution obtained using DC drbohlav2005optimal and LIPIDS (ours) for $M = 3,5,10,20$.