SpiderCam: Low-Power Snapshot Depth from Differential Defocus

Abstract

We introduce SpiderCam, an FPGA-based snapshot depth-from-defocus camera which produces 480x400 sparse depth maps in real-time at 32.5 FPS over a working range of 52 cm while consuming 624 mW of power in total. SpiderCam comprises a custom camera that simultaneously captures two differently focused images of the same scene, processed with a SystemVerilog implementation of depth from differential defocus (DfDD) on a low-power FPGA. To achieve state-of-the-art power consumption, we present algorithmic improvements to DfDD that overcome challenges caused by low-power sensors, and design a memory-local implementation for streaming depth computation on a device that is too small to store even a single image pair. We report the first sub-Watt total power measurement for passive FPGA-based 3D cameras in the literature.

Figures (15)

• Figure 1: Method overview. Our camera uses a beam splitter and a pair of differentially-defocused low power sensors to observe the same scene with an offset depth of field. Our algorithm processes these images to produce depth maps thresholded by confidence.
• Figure 2: Captured images and depth maps computed on-device. Note the handling of continuous depths (vs. disparity levels), reflective textures, and transparent objects that active sensing would struggle with, as well as objects in motion due to snapshot capture.
• Figure 3: Quantitative analysis on images from device. (a) Mean absolute error (MAE) of depth as a function of true depth, with working range bounds. Our method robustly outperforms the Focal Split algorithm luo2025focal in the low power hardware setting, and accounting for spatial variation in our compact optics is vital. Purple dots show errors from depth maps computed on-device and demonstrate good agreement with off-line depth computations (rest of figure), which were performed in 32-bit floating point. (b) The MAE vs. depth curve of our method with dynamic sparsity values set by per-image confidence percentiles (grays) and the MAE (solid purple) and output pixel density (100-sparsity, dashed purple) resulting from the fixed confidence thresholds used in our method. (c) Working range increases as data is sparsified by per-image confidence percentiles.
• Figure 4: Power/accuracy trade-offs for alternate algorithms. We estimate max core power on the ECP5 with manufacturer's tools and evaluate working range and MAE from 0.4m to 1.0m offline on images collected by our hardware prototype. Adding computational components improves accuracy while increasing power cost. Our algorithm is shown with the purple star.
• Figure 5: Resource Usage Across Methods. We compare the resource utilization of several configurations of our system as well as systems in the literature. "DS" and "SS" denote evaluation at 2 and 1 scales, respectively.