Inter-event Interval Microscopy for Event Cameras

TL;DR

This work tackles reconstructing meaningful optical information from sparse event streams in fluorescence microscopy by introducing Inter-event Interval Microscopy (IEIM). IEIM uses pulsed excitation to encode sample density into inter-event intervals, with the density $D(x,y)$ inversely related to the inter-event interval via $D(x,y)=\frac{\theta C_p}{H K\,(t_{(x,y,k)}-t_{(x,y,k-1)})}$; a regime change is described for high illumination levels. Hardware-wise, a pulse-modulation device (e.g., AOTF) is integrated into the microscope to realize high-frequency, low-amplitude excitation, and the method is validated on the IEIMat dataset, showing superior spatial/temporal resolution, higher dynamic range, and lower bandwidth than prior methods. The results indicate that IEIM can achieve real-time-like, high-fidelity fluorescence imaging for both static and dynamic scenes, with practical implications for rapid, high-DR microscopy and open data sharing via the IEIMat dataset.

Abstract

Event cameras, an innovative bio-inspired sensor, differ from traditional cameras by sensing changes in intensity rather than directly perceiving intensity and recording these variations as a continuous stream of "events". The intensity reconstruction from these sparse events has long been a challenging problem. Previous approaches mainly focused on transforming motion-induced events into videos or achieving intensity imaging for static scenes by integrating modulation devices at the event camera acquisition end. In this paper, for the first time, we achieve event-to-intensity conversion using a static event camera for both static and dynamic scenes in fluorescence microscopy. Unlike conventional methods that primarily rely on event integration, the proposed Inter-event Interval Microscopy (IEIM) quantifies the time interval between consecutive events at each pixel. With a fixed threshold in the event camera, the time interval can precisely represent the intensity. At the hardware level, the proposed IEIM integrates a pulse light modulation device within a microscope equipped with an event camera, termed Pulse Modulation-based Event-driven Fluorescence Microscopy. Additionally, we have collected IEIMat dataset under various scenes including high dynamic range and high-speed scenarios. Experimental results on the IEIMat dataset demonstrate that the proposed IEIM achieves superior spatial and temporal resolution, as well as a higher dynamic range, with lower bandwidth compared to other methods. The code and the IEIMat dataset will be made publicly available.

Figures (7)

• Figure 1: Event-based imaging of static and dynamic scenes. Our method achieves event-based imaging of static and dynamic scenes with quality that significantly exceeds state-of-the-art methods, offering higher temporal resolution and an extended dynamic range.
• Figure 2: Pipeline of IEIM. (a) The IEIM data collection device. It employs a periodic pulsed modulation of light intensity with a period T. The collected events are processed according to the (b) Inter-Event Interval principle, where intervals between events reflect the intensity. (c) IEI calculations are performed on all event streams, and an image is selected from each cycle to represent the intensity at that specific moment.
• Figure 3: Comparison in synthesized static scenes. Our method significantly outperforms SOTA methods.
• Figure 4: Comparison in synthesized motion scenes. Our method significantly outperforms SOTA methods.
• Figure 5: Comparison in real-world static scenes. Imaging with IEIM requires appropriate calibration between modulation frequency and light power and the resulting image quality far exceeds that of SOTA methods. The power in the lower right corner corresponds to the laser power at the output of the objective lens.