Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

WinMamba: Multi-Scale Shifted Windows in State Space Model for 3D Object Detection

Longhui Zheng, Qiming Xia, Xiaolu Chen, Zhaoliang Liu, Chenglu Wen

TL;DR

WinMamba addresses the trade-off between computational efficiency and long-range spatial modeling in 3D object detection by introducing a Mamba-based backbone with two novel modules: Window Shift Fusion (WSF) and Adaptive Window Fusion (AWF). WSF interleaves 3D window partitions to recover cross-window context, while AWF builds a higher-resolution auxiliary path with adaptive window sizes to preserve fine details across scales. The approach is validated on KITTI and Waymo Open Dataset, showing improved mAP, especially for small objects, with ablations confirming the contributions of WSF and AWF. The work advances efficient, scalable 3D feature encoding for autonomous driving and provides a publicly available codebase.

Abstract

3D object detection is critical for autonomous driving, yet it remains fundamentally challenging to simultaneously maximize computational efficiency and capture long-range spatial dependencies. We observed that Mamba-based models, with their linear state-space design, capture long-range dependencies at lower cost, offering a promising balance between efficiency and accuracy. However, existing methods rely on axis-aligned scanning within a fixed window, inevitably discarding spatial information. To address this problem, we propose WinMamba, a novel Mamba-based 3D feature-encoding backbone composed of stacked WinMamba blocks. To enhance the backbone with robust multi-scale representation, the WinMamba block incorporates a window-scale-adaptive module that compensates voxel features across varying resolutions during sampling. Meanwhile, to obtain rich contextual cues within the linear state space, we equip the WinMamba layer with a learnable positional encoding and a window-shift strategy. Extensive experiments on the KITTI and Waymo datasets demonstrate that WinMamba significantly outperforms the baseline. Ablation studies further validate the individual contributions of the WSF and AWF modules in improving detection accuracy. The code will be made publicly available.

WinMamba: Multi-Scale Shifted Windows in State Space Model for 3D Object Detection

TL;DR

WinMamba addresses the trade-off between computational efficiency and long-range spatial modeling in 3D object detection by introducing a Mamba-based backbone with two novel modules: Window Shift Fusion (WSF) and Adaptive Window Fusion (AWF). WSF interleaves 3D window partitions to recover cross-window context, while AWF builds a higher-resolution auxiliary path with adaptive window sizes to preserve fine details across scales. The approach is validated on KITTI and Waymo Open Dataset, showing improved mAP, especially for small objects, with ablations confirming the contributions of WSF and AWF. The work advances efficient, scalable 3D feature encoding for autonomous driving and provides a publicly available codebase.

Abstract

3D object detection is critical for autonomous driving, yet it remains fundamentally challenging to simultaneously maximize computational efficiency and capture long-range spatial dependencies. We observed that Mamba-based models, with their linear state-space design, capture long-range dependencies at lower cost, offering a promising balance between efficiency and accuracy. However, existing methods rely on axis-aligned scanning within a fixed window, inevitably discarding spatial information. To address this problem, we propose WinMamba, a novel Mamba-based 3D feature-encoding backbone composed of stacked WinMamba blocks. To enhance the backbone with robust multi-scale representation, the WinMamba block incorporates a window-scale-adaptive module that compensates voxel features across varying resolutions during sampling. Meanwhile, to obtain rich contextual cues within the linear state space, we equip the WinMamba layer with a learnable positional encoding and a window-shift strategy. Extensive experiments on the KITTI and Waymo datasets demonstrate that WinMamba significantly outperforms the baseline. Ablation studies further validate the individual contributions of the WSF and AWF modules in improving detection accuracy. The code will be made publicly available.

Paper Structure

This paper contains 15 sections, 10 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. 3D Object Detection in Point Clouds.
  4. Mamba in the Vision Task.
  5. Method
  6. Mamba-Based 3D Feature-Encoding Backbone
  7. Adaptive Window Fusion (AWF)
  8. Window Shift Fusion (WSF)
  9. Experiments
  10. Datasets and Metrics
  11. Implementation Details
  12. Main Results
  13. Ablation Study
  14. Conclusion
  15. Acknowledgements.

Figures (3)

  • Figure 1: Motivation & Solution (a) causes spatial fragmentation during serialization due to fixed-window partitioning. (b) recovers cross-window contextual information by WSF. (c) aligns window scales with feature pyramid and preserves semantic coherence within windows through AWF.
  • Figure 2: (a) illustrates the structure of WinMamba, composed of N stacked WinMamba Blocks. The lower path functions as the main path (FPN), while the upper path serves as the auxiliary path. Between the two paths, Window-Scale Adaptation (WSA) dynamically assigns suitable window sizes to the auxiliary path. (b) details the WinMamba layer, a feature extraction module incorporating learnable positional embeddings. The input features undergo alternating window partitioning along the X- and Y-axis, then be processed by the Mamba Block. (c) presents the Window Shift Fusion process. Voxel-wise features are serialized from both pre- and post-shift window partitions, encoded by the Mamba block, and subsequently fused.
  • Figure 3: Illustration of the scale alignment process between Main Path and Auxiliary Path across different feature levels.