Video Denoising in Fluorescence Guided Surgery

TL;DR

An accurate noise simulation pipeline that includes laser leakage light (LLL) is proposed and three baseline deep learning based algorithms for FGS video denoising are proposed.

Abstract

Fluorescence guided surgery (FGS) is a promising surgical technique that gives surgeons a unique view of tissue that is used to guide their practice by delineating tissue types and diseased areas. As new fluorescent contrast agents are developed that have low fluorescent photon yields, it becomes increasingly important to develop computational models to allow FGS systems to maintain good video quality in real time environments. To further complicate this task, FGS has a difficult bias noise term from laser leakage light (LLL) that represents unfiltered excitation light that can be on the order of the fluorescent signal. Most conventional video denoising methods focus on zero mean noise, and non-causal processing, both of which are violated in FGS. Luckily in FGS, often a co-located reference video is also captured which we use to simulate the LLL and assist in the denoising processes. In this work, we propose an accurate noise simulation pipeline that includes LLL and propose three baseline deep learning based algorithms for FGS video denoising.

Figures (14)

• Figure 1: Measurement and Noise: (a) In the FGS measurement process, excitation laser light and reference light shine onto the scene where the excitation light produces fluorescence at a higher wavelength. Three relevant spectral bands are imaged by the FGS system. The reference band (yellow) is isolated using a dichoric beamsplitter and imaged by the RV camera. The excitation laser band (red) is attenuated by the emission filter leaving the laser leakage light (LLL) which is combined with the fluorescence band (pink) and imaged on the FV camera. (b) In our noise simulation, we combine a clean fluorescence image with an LLL prediction from our LLL-PN. Then we apply Poisson noise and add sampled read noise frames to produce the final noisy FV. (c) shows the results of different denoisers on a noisy FV. State-of-the-art (SOTA) video denoisers struggle while our proposed BL-RNN performs well on this task.
• Figure 1: Gain Calibration Images: This figure shows the (a) mean, (b) variance, and (c) the mean divided by the variance of the Quel calibration phantom from OL-Phantom used to calibrate for our $K$ parameter. (d) shows the histogram of mean divided by variance values for the 9 wells. The mean value of this histogram is our calibrated $K$ value. Notice that the mean divided by variance values are similar in all wells indicating that a Poisson distribution is correct for the brightness level of this phantom.
• Figure 2: Dataset Example Images: Here we show two example images for both OL-2023 and OL-2024. OL-2023 focuses on vasculature where as OL-2024 focuses on local fluorescent regions.
• Figure 2: Flicker Noise: (a) shows the histogram of the per-frame average values of the OL-Dark dataset. The read noise pattern of our camera sensor exhibits strong flicker noise leading to a bimodal distribution, one example from each peak is shown. (b) shows the per-frame average of the OL-LLL dataset (green), the LLL-PN (orange), and the difference between the two (blue). Due to the bimodal flicker noise our LLL-PN predicts the "high" mode of the flicker; however, we are able to obtain the bimodal distribution when subtracting out our background indicating strong performance with subtracting out the LLL and leaving the read noise.
• Figure 3: Real and Simulated Data: (a) frames from our OL-Real test set. (b) simulated frames with increasing signal levels and fixed $L_{m}=50$. Qualitatively, $S_{m}=L_{m}=50$ closely matches many of the real data frames. Notice the hand (red arrow) has similar signal levels in both the real and simulated data at these parameters. (c) simulated frames with increasing $L_m$ and $S{m}=50$. Notice that the small fluorescent features (green arrow) fade as $L_m$ increases.