StereoMamba: Real-time and Robust Intraoperative Stereo Disparity Estimation via Long-range Spatial Dependencies

TL;DR

StereoMamba targets real-time, robust stereo disparity estimation for robot-assisted minimally invasive surgery by introducing FE-Mamba, a feature extractor that combines self- and cross-attention, and MFF, a fusion module that integrates multi-scale cross-image information. The method constructs a group-wise cost volume via cross-attention–driven features and a cost-volume decoder, trained with a multi-output smooth L1 loss and soft-argmin disparity estimation. On the SCARED benchmark, StereoMamba achieves state-of-the-art depth accuracy (EPE $2.64$ px, depth MAE $2.55$ mm) while maintaining real-time inference at $21.28$ FPS on high-resolution inputs, and shows strong zero-shot generalization to in-vivo datasets with high SSIM/PSNR. Ablation confirms FE-Mamba and MFF provide clear gains in accuracy and speed, underscoring the importance of long-range spatial dependencies and cross-image fusion for RAMIS disparity estimation. Overall, the approach delivers a balanced solution that combines accuracy, robustness, and efficiency suitable for real-world RAMIS deployment.

Abstract

Stereo disparity estimation is crucial for obtaining depth information in robot-assisted minimally invasive surgery (RAMIS). While current deep learning methods have made significant advancements, challenges remain in achieving an optimal balance between accuracy, robustness, and inference speed. To address these challenges, we propose the StereoMamba architecture, which is specifically designed for stereo disparity estimation in RAMIS. Our approach is based on a novel Feature Extraction Mamba (FE-Mamba) module, which enhances long-range spatial dependencies both within and across stereo images. To effectively integrate multi-scale features from FE-Mamba, we then introduce a novel Multidimensional Feature Fusion (MFF) module. Experiments against the state-of-the-art on the ex-vivo SCARED benchmark demonstrate that StereoMamba achieves superior performance on EPE of 2.64 px and depth MAE of 2.55 mm, the second-best performance on Bad2 of 41.49% and Bad3 of 26.99%, while maintaining an inference speed of 21.28 FPS for a pair of high-resolution images (1280*1024), striking the optimum balance between accuracy, robustness, and efficiency. Furthermore, by comparing synthesized right images, generated from warping left images using the generated disparity maps, with the actual right image, StereoMamba achieves the best average SSIM (0.8970) and PSNR (16.0761), exhibiting strong zero-shot generalization on the in-vivo RIS2017 and StereoMIS datasets.

Figures (4)

• Figure 1: Comparisons with SOTA disparity estimation methods. (a) shows the different methodologies between GwcNetguoGroupwiseCorrelationStereo2019, ACVNetxuAttentionConcatenationVolume2022, and our proposed StereoMamba. (b) illustrates the trade-off between accuracy and inference speed. (c) compares the similarity between synthesized right images (generated from left images and disparity maps on two unseen in-vivo datasets) and actual right images.
• Figure 2: The overall architecture of StereoMamba, which consists of the FE-Mamba module for feature extraction, the cost volume construction via the MFF module, and the cost volume aggregation for disparity estimation.
• Figure 3: Qualitative results on SCARED. The first column indicates the rectified left image, the second column indicates the ground truth disparity, and the other columns are estimated disparity maps from GwcNetguoGroupwiseCorrelationStereo2019, ACVNetxuAttentionConcatenationVolume2022, RAFT-StereolipsonRAFTStereoMultilevelRecurrent2021, IGEV-StereoxuIterativeGeometryEncoding2023, Selective-Stereowang2024selective, MSDESISpsychogyiosMSDESISMultitaskStereo2022, Shi et al.shiBidirectionalSemiSupervisedDualBranch2023 and our StereoMamba. All methods share the same disparity colorbar, ranging from 0 to 192 pixels. Additional visualization results are provided in the supplementary material.
• Figure 4: Qualitative results on StereoMIS (first 3 rows) and RIS_2017 (last 3 rows). The first column indicates the rectified left image, the other columns are estimated disparity maps from GwcNetguoGroupwiseCorrelationStereo2019, ACVNetxuAttentionConcatenationVolume2022, RAFT-StereolipsonRAFTStereoMultilevelRecurrent2021, IGEV-StereoxuIterativeGeometryEncoding2023, Selective-Stereowang2024selective, MSDESISpsychogyiosMSDESISMultitaskStereo2022, Shi et al.shiBidirectionalSemiSupervisedDualBranch2023 and our StereoMamba. All methods share the same disparity colorbar, ranging from 0 to 192 pixels. Additional visualization results are provided in supplementary material.