Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

VMambaCC: A Visual State Space Model for Crowd Counting

Hao-Yuan Ma, Li Zhang, Shuai Shi

TL;DR

This paper introduces VMambaCC, a visual state space model for crowd counting that achieves global perception with linear complexity. It combines a VMamba-based backbone, a Multi-head High-level Feature (MHF) attention mechanism, and a High-level Semantic Supervised Feature Pyramid Network (HS2FPN) to fuse high- and low-level features for robust counting and localization. The approach employs a Three-Task Combination (TTC) loss with Hungarian matching to align annotated and predicted points, and demonstrates strong performance across Shanghai Tech, UCF_QNRF, JHU-Crowd, and UCF_CC_50 datasets, including state-of-the-art results on several metrics. The results indicate that integrating high-level semantic guidance into multi-scale fusion yields substantial gains in accuracy and localization, with practical implications for efficient crowd monitoring and analysis.

Abstract

As a deep learning model, Visual Mamba (VMamba) has a low computational complexity and a global receptive field, which has been successful applied to image classification and detection. To extend its applications, we apply VMamba to crowd counting and propose a novel VMambaCC (VMamba Crowd Counting) model. Naturally, VMambaCC inherits the merits of VMamba, or global modeling for images and low computational cost. Additionally, we design a Multi-head High-level Feature (MHF) attention mechanism for VMambaCC. MHF is a new attention mechanism that leverages high-level semantic features to augment low-level semantic features, thereby enhancing spatial feature representation with greater precision. Building upon MHF, we further present a High-level Semantic Supervised Feature Pyramid Network (HS2PFN) that progressively integrates and enhances high-level semantic information with low-level semantic information. Extensive experimental results on five public datasets validate the efficacy of our approach. For example, our method achieves a mean absolute error of 51.87 and a mean squared error of 81.3 on the ShangHaiTech\_PartA dataset. Our code is coming soon.

VMambaCC: A Visual State Space Model for Crowd Counting

TL;DR

This paper introduces VMambaCC, a visual state space model for crowd counting that achieves global perception with linear complexity. It combines a VMamba-based backbone, a Multi-head High-level Feature (MHF) attention mechanism, and a High-level Semantic Supervised Feature Pyramid Network (HS2FPN) to fuse high- and low-level features for robust counting and localization. The approach employs a Three-Task Combination (TTC) loss with Hungarian matching to align annotated and predicted points, and demonstrates strong performance across Shanghai Tech, UCF_QNRF, JHU-Crowd, and UCF_CC_50 datasets, including state-of-the-art results on several metrics. The results indicate that integrating high-level semantic guidance into multi-scale fusion yields substantial gains in accuracy and localization, with practical implications for efficient crowd monitoring and analysis.

Abstract

As a deep learning model, Visual Mamba (VMamba) has a low computational complexity and a global receptive field, which has been successful applied to image classification and detection. To extend its applications, we apply VMamba to crowd counting and propose a novel VMambaCC (VMamba Crowd Counting) model. Naturally, VMambaCC inherits the merits of VMamba, or global modeling for images and low computational cost. Additionally, we design a Multi-head High-level Feature (MHF) attention mechanism for VMambaCC. MHF is a new attention mechanism that leverages high-level semantic features to augment low-level semantic features, thereby enhancing spatial feature representation with greater precision. Building upon MHF, we further present a High-level Semantic Supervised Feature Pyramid Network (HS2PFN) that progressively integrates and enhances high-level semantic information with low-level semantic information. Extensive experimental results on five public datasets validate the efficacy of our approach. For example, our method achieves a mean absolute error of 51.87 and a mean squared error of 81.3 on the ShangHaiTech\_PartA dataset. Our code is coming soon.
Paper Structure (16 sections, 13 equations, 5 figures, 8 tables)

This paper contains 16 sections, 13 equations, 5 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Proposed model
  4. Framework of VMambaCC
  5. MHF attention mechanism
  6. CEM
  7. MSEM
  8. HCEM
  9. HS2FPN
  10. Experiments
  11. Experimental Setups
  12. Counting Experiments
  13. Localization Experiments
  14. Ablation experiments
  15. Visualization
  16. ...and 1 more sections

Figures (5)

  • Figure 1: Comparison curves of flops vs. image size obtained by four methods.
  • Figure 2: Architecture of VMambaCC model.
  • Figure 3: Structure of MHF attention mechanism.
  • Figure 4: MHF Attention block connection.
  • Figure 5: Demonstration of the predictive effect of our model.