vGamba: Attentive State Space Bottleneck for efficient Long-range Dependencies in Visual Recognition

TL;DR

vGamba presents a lightweight, SSM-enhanced bottleneck for vision that merges a 2D-adapted Mamba-based Gamba Cell with a 2D-relative positional encoding and a gating-attention ASC module to efficiently capture long-range dependencies. Replacing a 3×3 convolution in a CNN bottleneck, it achieves competitive accuracy with lower FLOPs and smaller parameter counts, outperforming several CNN and Transformer baselines across ImageNet, ADE20K, and COCO. The work demonstrates that a single forward SSM pass, coupled with targeted spatial attention, can bridge the gap between CNNs and Transformers, delivering scalable, high-performance backbones for classification, detection, and segmentation. This approach offers practical benefits for real-world, high-resolution vision systems and provides a template for integrating SSMs into standard CNN pipelines with minimal architectural overhead.

Abstract

Capturing long-range dependencies efficiently is essential for visual recognition tasks, yet existing methods face limitations. Convolutional neural networks (CNNs) struggle with restricted receptive fields, while Vision Transformers (ViTs) achieve global context and long-range modeling at a high computational cost. State-space models (SSMs) offer an alternative, but their application in vision remains underexplored. This work introduces vGamba, a hybrid vision backbone that integrates SSMs with attention mechanisms to enhance efficiency and expressiveness. At its core, the Gamba bottleneck block that includes, Gamba Cell, an adaptation of Mamba for 2D spatial structures, alongside a Multi-Head Self-Attention (MHSA) mechanism and a Gated Fusion Module for effective feature representation. The interplay of these components ensures that vGamba leverages the low computational demands of SSMs while maintaining the accuracy of attention mechanisms for modeling long-range dependencies in vision tasks. Additionally, the Fusion module enables seamless interaction between these components. Extensive experiments on classification, detection, and segmentation tasks demonstrate that vGamba achieves a superior trade-off between accuracy and computational efficiency, outperforming several existing models.

Figures (8)

• Figure 1: Bottleneck evolution from CNN based, ViT, and SSM
• Figure 2: To test the models’ ability to capture LRD, we transport information between distant spatial diagonal locations, placing the source and target in opposite corners using different shapes. We compare three bottleneck architectures ResNet-50 (CNN), BotNet (Transformer), and vGamba (SSM) all trained under identical settings (AdamW, lr 0.001, 50 epochs, $224^2$, MSE). ResNet-50 shows an almost straight-line convergence curve, indicating limited capacity to model LRD. In contrast, vGamba converges faster and more smoothly, producing more accurate reconstructions than ResNet-50 and BotNet, effectively capturing LRD and spatial transport patterns.
• Figure 3: Schematic comparison of the proposed Gamba cell block (c): (a) Vmamba block liu2025vmamba, and (b) ViM block zhu2024vision. Our approach introduces a single SSM layer that can directly replace the 3×3 conv. in a standard CNN bottleneck, efficiently modeling LRD. In contrast, VMamba requires four replicas of the SSM per block, and ViM requires two replicas.
• Figure 4: Gamba Bottleneck showing Gamba Cell and the ASC module.
• Figure 5: ASC fuses SSM and RPE to form the Gamba cell.