Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SEDMamba: Enhancing Selective State Space Modelling with Bottleneck Mechanism and Fine-to-Coarse Temporal Fusion for Efficient Error Detection in Robot-Assisted Surgery

Jialang Xu, Nazir Sirajudeen, Matthew Boal, Nader Francis, Danail Stoyanov, Evangelos Mazomenos

TL;DR

This paper tackles automated surgical error detection in robotic-assisted surgery by addressing the need for long-range temporal modeling with high efficiency. It introduces SEDMamba, a hierarchical model that combines selective state space models with a bottleneck mechanism and fine-to-coarse temporal fusion to capture multi-scale temporal context in long videos. A new frame-level in-vivo error dataset for SAR-RARP50 annotated with OCHRA provides a strong benchmark, and SEDMamba achieves state-of-the-art AUC and AP gains while dramatically reducing parameters and FLOPs. The approach demonstrates practical impact for real surgical cases and offers a roadmap for leveraging frame-level annotations in long-sequence video understanding of surgical errors.

Abstract

Automated detection of surgical errors can improve robotic-assisted surgery. Despite promising progress, existing methods still face challenges in capturing rich temporal context to establish long-term dependencies while maintaining computational efficiency. In this paper, we propose a novel hierarchical model named SEDMamba, which incorporates the selective state space model (SSM) into surgical error detection, facilitating efficient long sequence modelling with linear complexity. SEDMamba enhances selective SSM with a bottleneck mechanism and fine-to-coarse temporal fusion (FCTF) to detect and temporally localize surgical errors in long videos. The bottleneck mechanism compresses and restores features within their spatial dimension, thereby reducing computational complexity. FCTF utilizes multiple dilated 1D convolutional layers to merge temporal information across diverse scale ranges, accommodating errors of varying duration. Our work also contributes the first-of-its-kind, frame-level, in-vivo surgical error dataset to support error detection in real surgical cases. Specifically, we deploy the clinically validated observational clinical human reliability assessment tool (OCHRA) to annotate the errors during suturing tasks in an open-source radical prostatectomy dataset (SAR-RARP50). Experimental results demonstrate that our SEDMamba outperforms state-of-the-art methods with at least 1.82% AUC and 3.80% AP performance gains with significantly reduced computational complexity. The corresponding error annotations, code and models are released at https://github.com/wzjialang/SEDMamba.

SEDMamba: Enhancing Selective State Space Modelling with Bottleneck Mechanism and Fine-to-Coarse Temporal Fusion for Efficient Error Detection in Robot-Assisted Surgery

TL;DR

This paper tackles automated surgical error detection in robotic-assisted surgery by addressing the need for long-range temporal modeling with high efficiency. It introduces SEDMamba, a hierarchical model that combines selective state space models with a bottleneck mechanism and fine-to-coarse temporal fusion to capture multi-scale temporal context in long videos. A new frame-level in-vivo error dataset for SAR-RARP50 annotated with OCHRA provides a strong benchmark, and SEDMamba achieves state-of-the-art AUC and AP gains while dramatically reducing parameters and FLOPs. The approach demonstrates practical impact for real surgical cases and offers a roadmap for leveraging frame-level annotations in long-sequence video understanding of surgical errors.

Abstract

Automated detection of surgical errors can improve robotic-assisted surgery. Despite promising progress, existing methods still face challenges in capturing rich temporal context to establish long-term dependencies while maintaining computational efficiency. In this paper, we propose a novel hierarchical model named SEDMamba, which incorporates the selective state space model (SSM) into surgical error detection, facilitating efficient long sequence modelling with linear complexity. SEDMamba enhances selective SSM with a bottleneck mechanism and fine-to-coarse temporal fusion (FCTF) to detect and temporally localize surgical errors in long videos. The bottleneck mechanism compresses and restores features within their spatial dimension, thereby reducing computational complexity. FCTF utilizes multiple dilated 1D convolutional layers to merge temporal information across diverse scale ranges, accommodating errors of varying duration. Our work also contributes the first-of-its-kind, frame-level, in-vivo surgical error dataset to support error detection in real surgical cases. Specifically, we deploy the clinically validated observational clinical human reliability assessment tool (OCHRA) to annotate the errors during suturing tasks in an open-source radical prostatectomy dataset (SAR-RARP50). Experimental results demonstrate that our SEDMamba outperforms state-of-the-art methods with at least 1.82% AUC and 3.80% AP performance gains with significantly reduced computational complexity. The corresponding error annotations, code and models are released at https://github.com/wzjialang/SEDMamba.
Paper Structure (21 sections, 7 equations, 4 figures, 8 tables)

This paper contains 21 sections, 7 equations, 4 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Dataset Description and Annotation
  4. Surgical Error Detection Model (SEDMamba)
  5. Preliminaries
  6. Overview of SEDMamba
  7. Bottleneck Multi-scale State Space (BMSS) Block
  8. Fine-to-Coarse Temporal Fusion (FCTF)
  9. Experiments
  10. Experimental Settings
  11. Evaluation Metrics
  12. Comparison with the State-of-the-Art
  13. Computational Efficiency
  14. Ablation Study
  15. Effectiveness of Key Components in BMSS Block
  16. ...and 6 more sections

Figures (4)

  • Figure 1: A surgical video from the SAR-RARP50 dataset, including errors of varying duration.
  • Figure 2: Example error annotations at SAR-RARP50: (a) E6-Incorrect angle grasping needle; (b) E5-Tissue damage; (c) E24-Poor instrument control (clashing).
  • Figure 3: The pipeline of the proposed SEDMamba. (a) The overall architecture of SEDMamba. (b) The fundamental block of SEDMamba, namely the bottleneck multi-scale state space (BMSS) block; (c) Fine-to-coarse temporal fusion (FCTF); (d) Bottleneck mechanism.
  • Figure 4: Visualization results of the proposed SEDMamba (green line) and the second-best method Mamba (blue line). Each row represents one complete test video. The X-axis is the frame index while the Y-axis is the error probability output by the models. The red translucent background indicates error frames, while the white background indicates normal frames, i.e. ground truth of each frame. Different error types are considered, along with the corresponding prediction results (i.e., error probability). Red and black boxes are used to highlight significant results.