Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MambaOcc: Visual State Space Model for BEV-based Occupancy Prediction with Local Adaptive Reordering

Yonglin Tian, Songlin Bai, Zhiyao Luo, Yutong Wang, Yisheng Lv, Fei-Yue Wang

TL;DR

MambaOcc introduces a lightweight BEV-based occupancy predictor built on a Mamba-state-space backbone and a local adaptive reordering (LAR) strategy to enable efficient long-range perception. The method integrates VMamba features, BEV view transformation, and a hybrid LAR-SS2D encoder to produce accurate 3D occupancy predictions with far fewer parameters and FLOPs than Transformer-based rivals. Key contributions include the LAR mechanism with anchors and many-to-one mappings, and a temporal fusion module that leverages ego-motion to enhance context. On Occ3D-nuScenes, MambaOcc achieves state-of-the-art accuracy while significantly reducing computational burden, highlighting practical potential for real-time autonomous driving systems.

Abstract

Occupancy prediction has attracted intensive attention and shown great superiority in the development of autonomous driving systems. The fine-grained environmental representation brought by occupancy prediction in terms of both geometry and semantic information has facilitated the general perception and safe planning under open scenarios. However, it also brings high computation costs and heavy parameters in existing works that utilize voxel-based 3d dense representation and Transformer-based quadratic attention. To address these challenges, in this paper, we propose a Mamba-based occupancy prediction method (MambaOcc) adopting BEV features to ease the burden of 3D scenario representation, and linear Mamba-style attention to achieve efficient long-range perception. Besides, to address the sensitivity of Mamba to sequence order, we propose a local adaptive reordering (LAR) mechanism with deformable convolution and design a hybrid BEV encoder comprised of convolution layers and Mamba. Extensive experiments on the Occ3D-nuScenes dataset demonstrate that MambaOcc achieves state-of-the-art performance in terms of both accuracy and computational efficiency. For example, compared to FlashOcc, MambaOcc delivers superior results while reducing the number of parameters by 42\% and computational costs by 39\%. Code will be available at https://github.com/Hub-Tian/MambaOcc.

MambaOcc: Visual State Space Model for BEV-based Occupancy Prediction with Local Adaptive Reordering

TL;DR

MambaOcc introduces a lightweight BEV-based occupancy predictor built on a Mamba-state-space backbone and a local adaptive reordering (LAR) strategy to enable efficient long-range perception. The method integrates VMamba features, BEV view transformation, and a hybrid LAR-SS2D encoder to produce accurate 3D occupancy predictions with far fewer parameters and FLOPs than Transformer-based rivals. Key contributions include the LAR mechanism with anchors and many-to-one mappings, and a temporal fusion module that leverages ego-motion to enhance context. On Occ3D-nuScenes, MambaOcc achieves state-of-the-art accuracy while significantly reducing computational burden, highlighting practical potential for real-time autonomous driving systems.

Abstract

Occupancy prediction has attracted intensive attention and shown great superiority in the development of autonomous driving systems. The fine-grained environmental representation brought by occupancy prediction in terms of both geometry and semantic information has facilitated the general perception and safe planning under open scenarios. However, it also brings high computation costs and heavy parameters in existing works that utilize voxel-based 3d dense representation and Transformer-based quadratic attention. To address these challenges, in this paper, we propose a Mamba-based occupancy prediction method (MambaOcc) adopting BEV features to ease the burden of 3D scenario representation, and linear Mamba-style attention to achieve efficient long-range perception. Besides, to address the sensitivity of Mamba to sequence order, we propose a local adaptive reordering (LAR) mechanism with deformable convolution and design a hybrid BEV encoder comprised of convolution layers and Mamba. Extensive experiments on the Occ3D-nuScenes dataset demonstrate that MambaOcc achieves state-of-the-art performance in terms of both accuracy and computational efficiency. For example, compared to FlashOcc, MambaOcc delivers superior results while reducing the number of parameters by 42\% and computational costs by 39\%. Code will be available at https://github.com/Hub-Tian/MambaOcc.
Paper Structure (22 sections, 4 equations, 5 figures, 8 tables)

This paper contains 22 sections, 4 equations, 5 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. 3D Occupancy Prediction.
  4. State Space Models.
  5. Method
  6. VM-Backbone
  7. View Transformation and Temporal Fusion
  8. LAR-SS2D Hybrid BEV Encoder
  9. Occupancy Prediction Head
  10. Experiments
  11. Dataset and Training Details
  12. Comparison with SOTA appropachs
  13. Ablation Studies
  14. Effectiveness of the Proposed Modules
  15. Initialization of Image Backbone.
  16. ...and 7 more sections

Figures (5)

  • Figure 1: The framework of MambaOcc. We present the network overview in a schematic form at the top of the figure, while the bottom part of the figure provides a structural diagram of the network.
  • Figure 2: The demo of occupancy prediction of MambaOcc.
  • Figure 3: The visualized comparison of occupancy prediction between MambaOcc and FlashOcc.
  • Figure 4: The visualization of one-to-one mapping (group 0 and group 1).
  • Figure 5: The visualization of one-to-one mapping (group 2 and group 3).