Event-based Photometric Stereo via Rotating Illumination and Per-Pixel Learning

TL;DR

An event-based photometric stereo system that leverages an event camera, which is effective in scenarios with continuously varying scene radiance and high dynamic range conditions, and introduces a lightweight per-pixel multi-layer neural network that directly predicts surface normals from event signals generated by intensity changes as the light source rotates, without system calibration.

Abstract

Photometric stereo is a technique for estimating surface normals using images captured under varying illumination. However, conventional frame-based photometric stereo methods are limited in real-world applications due to their reliance on controlled lighting, and susceptibility to ambient illumination. To address these limitations, we propose an event-based photometric stereo system that leverages an event camera, which is effective in scenarios with continuously varying scene radiance and high dynamic range conditions. Our setup employs a single light source moving along a predefined circular trajectory, eliminating the need for multiple synchronized light sources and enabling a more compact and scalable design. We further introduce a lightweight per-pixel multi-layer neural network that directly predicts surface normals from event signals generated by intensity changes as the light source rotates, without system calibration. Experimental results on benchmark datasets and real-world data collected with our data acquisition system demonstrate the effectiveness of our method, achieving a 7.12\% reduction in mean angular error compared to existing event-based photometric stereo methods. In addition, our method demonstrates robustness in regions with sparse event activity, strong ambient illumination, and scenes affected by specularities.

Figures (16)

• Figure 1: Overview of the proposed method. (a) Our system setup. A light source rotates around the optical axis of a fixed event camera. (b) Event signals observed over one rotation cycle of the light source. The signals are converted to a polarity-based event representation that encodes temporal illumination variations. (c) The polarity-based representation is aggregated at each pixel to form a polarity sum vector, which is used as the input to a surface normal estimation network. The network learns the nonlinear mapping from temporal event patterns to surface normals on a per-pixel basis and predicts dense surface normal maps without system calibration.
• Figure 2: Schematic diagram of our system. The system consists of a fixed event camera and a light source. The light source rotates around the optical axis of the event camera at a constant angular speed.
• Figure 3: Illustration of the process of generating event representations. (a) Intensity variations are induced by a rotating light source. (b) Events triggered by these variations are temporally sliced to compute the event representation.
• Figure 4: Comparison between simulated and real event data over one cycle of the light source rotation. (a) Event signals during one cycle. Blue points represent ON events, and red points represent OFF events. (b) Estimated surface normal maps and corresponding angular error maps for each event signal.
• Figure 5: Our experiment setting. The rotating platform consists of a DC motor, a timing belt, and a hollow disc. The disc is connected to the motor through the timing belt, and rotates together. The light source is attached to the edge of the disc.