Event-Based Structured Light for Depth Reconstruction using Frequency Tagged Light Patterns

TL;DR

The paper tackles real-time 3D depth estimation under challenging lighting by combining frequency-tagged structured-light patterns with an asynchronous event-based ATIS sensor and a high-speed DLP projector. It introduces three pattern-coding strategies (frequency/dutycycle, orientation tracking, and phase shifting) and supportive processing (burst filtering and random phase shifts) to robustly decode timing information for triangulation. Real-world experiments show depth reconstruction capabilities across a range of frequencies, yielding 3D point clouds, though real-time processing in MATLAB was not fully real-time in the reported setup. The approach offers a scalable path for high-speed, robust depth sensing in dynamic environments, with potential improvements in precise frequency extraction and simpler pattern designs as event-based sensors evolve.

Abstract

This paper presents a new method for 3D depth estimation using the output of an asynchronous time driven image sensor. In association with a high speed Digital Light Processing projection system, our method achieves real-time reconstruction of 3D points cloud, up to several hundreds of hertz. Unlike state of the art methodology, we introduce a method that relies on the use of frequency tagged light pattern that make use of the high temporal resolution of event based sensors. This approch eases matching as each pattern unique frequency allow for any easy matching between displayed patterns and the event based sensor. Results are show on real scenes.

Figures (11)

• Figure 1: Functional diagram of an ATIS pixel Posch2011. Two types of asynchronous events, encoding change and brightness information, are generated and transmitted individually by each pixel in the imaging array.
• Figure 2: The DLP lightcrafter module ®
• Figure 3: Experimental setup
• Figure 4: Structured light principle. A coded light pattern is projected onto the scene that is observed by a camera. Decoding the pattern allows the matching of paired points in the two views ($p_1, p_2$) and by triangulation of the rays coming from optical centers $C_1$ and $C_2$, the position of the real point $P$ is found. Colors in the picture are for clarity only and in our case refers to different binary signal's dutycycle for the first method or different motion orientation for the second.
• Figure 5: Example of the burst filter on a sinusoidal input. The input signal is transformed into an event stream by the sensor and at each new event the inter-spike delay is added to the filter's action potential. Between events, the AP follows an exponential decay. At any given polarity state, when the opposite polarity threshold is reached, the output toggles, effectively reflecting the changes of the input signal.