Photon Inhibition for Energy-Efficient Single-Photon Imaging

TL;DR

This work tackles the high energy cost of SPAD-based single-photon imaging by introducing photon inhibition, a lightweight, on-sensor strategy that adaptively disables SPAD pixels in space and time to reduce avalanche energy without severely compromising vision tasks. By formulating a formal observation model with inhibition and designing spatio-temporal inhibition policies, the authors derive energy-aware performance metrics and demonstrate, through simulations and real SPAD data, substantial energy savings (over 90% photon inhibition in some scenarios) while preserving image reconstruction, edge detection, and object-detection performance. The approach is supported by a suite of policies (including calculation-based and saturation look-ahead variants) that rely on simple local kernels and thresholds, enabling potential in-pixel implementation. The work advances energy-efficient single-photon imaging and opens avenues for hardware-aware designs that decouple flux from detections, with practical implications for high-speed, low-light imaging and embedded vision systems.

Abstract

Single-photon cameras (SPCs) are emerging as sensors of choice for various challenging imaging applications. One class of SPCs based on the single-photon avalanche diode (SPAD) detects individual photons using an avalanche process; the raw photon data can then be processed to extract scene information under extremely low light, high dynamic range, and rapid motion. Yet, single-photon sensitivity in SPADs comes at a cost -- each photon detection consumes more energy than that of a CMOS camera. This avalanche power significantly limits sensor resolution and could restrict widespread adoption of SPAD-based SPCs. We propose a computational-imaging approach called \emph{photon inhibition} to address this challenge. Photon inhibition strategically allocates detections in space and time based on downstream inference task goals and resource constraints. We develop lightweight, on-sensor computational inhibition policies that use past photon data to disable SPAD pixels in real-time, to select the most informative future photons. As case studies, we design policies tailored for image reconstruction and edge detection, and demonstrate, both via simulations and real SPC captured data, considerable reduction in photon detections (over 90\% of photons) while maintaining task performance metrics. Our work raises the question of ``which photons should be detected?'', and paves the way for future energy-efficient single-photon imaging.

Figures (17)

• Figure 1: Photon inhibition for resource-efficient passive SPAD imaging. (a) Unlike conventional CMOS cameras (CIS), the energy consumption in SPAD cameras increases with scene brightness, severely limiting the applicability of high resolution SPAD cameras in resource-constrained applications. (b) We expand the conventional imaging pipeline to incorporate "inhibition" that electronically enables or disables individual pixels to limit bandwidth and power consumption. Our method relies on lightweight mathematical operations called "inhibition policies" that update the inhibition patterns based on the history of photon detections. Inhibition policies can be optimized for image SNR or for downstream vision tasks.(c,d) Object detection, a high-level vision task, is successful even with a large fraction of photons inhibited.
• Figure 1: Power-efficient static single-photon imaging via inhibition. A reference image (BSDS500: 393035) displayed in the top left is captured using a bracket of three exposure times with 1,000 binary frames for each exposure time. The second column displays the resulting average inhibition patterns for each exposure time. The top most pattern from the shortest exposure time modestly inhibits and does so at the brightest pixels only. The inhibition pattern of the longest exposure time allocates most measurements to the darkest areas of the scene (in the shadows to the right of boat in the the foreground). The bottom chart summarizes the inhibition patterns using smoothed curves of the inhibition percent versus the flux of each pixel for each of the three exposure times (Lowess filter with a fraction of 1/5). The right-most columns show binary rate images using gamma compression ($\gamma = 0.4$) without (left) and with (right) inhibition at equal average detections per pixel. Detections increase moving down with averages of 5, 12, and 30 detections per pixel shown. Image quality metrics versus detections per pixel are summarized in the center and bottom of the left most column.
• Figure 2: Calculation-based inhibition with dead time. Arrows represent photons with an 'X' for inhibition. $T$ indicates the clocked recharge period. A score, $S$, calculated from past frames determines if future measurements are enabled or disabled.
• Figure 2: Power-efficient static single-photon imaging via inhibition. A reference image (BSDS500: 179084) displayed in the top left is captured using a bracket of three exposure times with 1,000 binary frames for each exposure time. The second column displays the resulting average inhibition patterns for each exposure time. The top most pattern from the shortest exposure time only modestly inhibits and does so at the brightest pixels only (maximum of $\sim$60% inhibition, primarily in the sky). The inhibition pattern of the longest exposure time allocates most measurements to the darkest areas of the scene (the hilltop and the dark areas of the helicopter). The bottom chart summarizes the inhibition patterns using smoothed curves of the inhibition percent versus the flux of each pixel for each of the three exposure times (Lowess filter with a fraction of 1/5). The right-most columns show binary rate images using gamma compression ($\gamma = 0.4$) without (left) and with (right) inhibition at equal average detections per pixel. Detections increase moving down with averages of 5, 12, and 30 detections per pixel shown. Image quality metrics versus detections per pixel are summarized in the center and bottom of the left most column.
• Figure 3: Efficiency metrics and inhibition policies that track the metrics: (a) The $\mathop{\mathrm{\mathsf{SNR}}}\nolimits_H$ in dB (black), the detection efficiency (red, $--$), and the measurement efficiency (blue, $- \cdot$$-$) versus the exposure with $W=100$ measurements. The binary rate $Y = 1 - e^{-H}$ is indicated on the top axis. The vertical dotted line indicates the exposure and the binary rate at which the $\mathop{\mathrm{\mathsf{SNR}}}\nolimits_H$ degrades by 3dB from the peak $\mathop{\mathrm{\mathsf{SNR}}}\nolimits_H$. (b,c) Monte Carlo simulations of binary images using calculation-based inhibition policies demonstrate how the allocation of measurements versus the pixel exposure depends upon tuning parameters. (b) the inhibition threshold $\eta$ adjusts the exposure level at which pixels are inhibited to allow the measurement fraction (the ratio of active measurements to total number of frames) to follow the $\mathop{\mathrm{\mathsf{SNR}}}\nolimits^2_{H/D}$ curve in (a). A smaller threshold more aggressively inhibits photons. (c) demonstrates the impact of the hold-off time, $\tau_H$, on the number of measurements allocated to the brightest pixels. The legend indicates the total fraction of photons inhibited as $I_F$. (b,c) show smoothed curves (Lowess filter, fraction of 1/5) of the measurement fraction vs. H.