Rethinking Surgical Smoke: A Smoke-Type-Aware Laparoscopic Video Desmoking Method and Dataset

TL;DR

This work addresses the challenge of visual degradation in laparoscopic videos caused by surgical smoke by introducing a smoke-type-aware desmoking framework (STANet) that distinguishes Diffusion Smoke and Ambient Smoke. STANet integrates three sub-networks: Smoky Feature Perception, Smoke Mask Segmentation with Semantic Soft Segmentation and coarse-to-fine disentanglement, and a Smokeless Video Reconstruction module that uses deformable and adaptive dilated convolutions guided by smoke masks. A large-scale synthetic STSVD dataset with smoke-type annotations supports training and evaluation, and comprehensive experiments demonstrate superior restoration quality and better generalization to downstream surgical tasks compared with state-of-the-art methods. The work delivers practical gains for real-time surgical guidance and downstream analyses, and provides a dataset to foster further research in smoke-aware desmoking.

Abstract

Electrocautery or lasers will inevitably generate surgical smoke, which hinders the visual guidance of laparoscopic videos for surgical procedures. The surgical smoke can be classified into different types based on its motion patterns, leading to distinctive spatio-temporal characteristics across smoky laparoscopic videos. However, existing desmoking methods fail to account for such smoke-type-specific distinctions. Therefore, we propose the first Smoke-Type-Aware Laparoscopic Video Desmoking Network (STANet) by introducing two smoke types: Diffusion Smoke and Ambient Smoke. Specifically, a smoke mask segmentation sub-network is designed to jointly conduct smoke mask and smoke type predictions based on the attention-weighted mask aggregation, while a smokeless video reconstruction sub-network is proposed to perform specially desmoking on smoky features guided by two types of smoke mask. To address the entanglement challenges of two smoke types, we further embed a coarse-to-fine disentanglement module into the mask segmentation sub-network, which yields more accurate disentangled masks through the smoke-type-aware cross attention between non-entangled and entangled regions. In addition, we also construct the first large-scale synthetic video desmoking dataset with smoke type annotations. Extensive experiments demonstrate that our method not only outperforms state-of-the-art approaches in quality evaluations, but also exhibits superior generalization across multiple downstream surgical tasks.

Figures (10)

• Figure 1: Visual comparison of two smoke types in surgical videos. From top to bottom, the rows represent smoky frames at successive time steps and their corresponding smoke masks.
• Figure 2: The overall framework of our STANet. The pink, blue and orange regions respectively denote the Smoky Feature Perception Sub-network, the Smoke Mask Segmentation Sub-network and the Smokeless Video Reconstruction Sub-network.
• Figure 3: The illustration of the Coarse-to-Fine Disentanglement Module(C2FDM).
• Figure 4: Statistics of the proposed STSVD dataset.
• Figure 5: The qualitative comparison between different methods. More results are given on the paper website.