Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TCFormer: A 5M-Parameter Transformer with Density-Guided Aggregation for Weakly-Supervised Crowd Counting

Qiang Guo, Rubo Zhang, Bingbing Zhang, Junjie Liu, Jianqing Liu

TL;DR

This paper tackles the challenge of high annotation costs and heavy computation in crowd counting by introducing TCFormer, a tiny transformer framework trained with only image-level counts. It combines a compact TinyViT backbone, a Learnable Density-Weighted Averaging module for density-aware spatial aggregation, and a dual-head supervision scheme with a count regression head and a density-level classification head, all optimized with a multi-task loss. On ShanghaiTech A/B, UCF-QNRF, and NWPU, TCFormer achieves competitive or superior accuracy while maintaining a small footprint (5M parameters) and modest compute (about 1.18 GFLOPs), outperforming many weakly supervised baselines and rivaling heavier supervised models. The approach also demonstrates practical edge deployment potential, with low GPU power consumption (~59 W) and favorable energy per image, illustrating that density-aware aggregation plus global context can bridge the gap between weak supervision and deployment on resource-constrained devices.

Abstract

Crowd counting typically relies on labor-intensive point-level annotations and computationally intensive backbones, restricting its scalability and deployment in resource-constrained environments. To address these challenges, this paper proposes the TCFormer, a tiny, ultra-lightweight, weakly-supervised transformer-based crowd counting framework with only 5 million parameters that achieves competitive performance. Firstly, a powerful yet efficient vision transformer is adopted as the feature extractor, the global context-aware capabilities of which provides semantic meaningful crowd features with a minimal memory footprint. Secondly, to compensate for the lack of spatial supervision, we design a feature aggregation mechanism termed the Learnable Density-Weighted Averaging module. This module dynamically re-weights local tokens according to predicted density scores, enabling the network to adaptively modulate regional features based on their specific density characteristics without the need for additional annotations. Furthermore, this paper introduces a density-level classification loss, which discretizes crowd density into distinct grades, thereby regularizing the training process and enhancing the model's classification power across varying levels of crowd density. Therefore, although TCformer is trained under a weakly-supervised paradigm utilizing only image-level global counts, the joint optimization of count and density-level losses enables the framework to achieve high estimation accuracy. Extensive experiments on four benchmarks including ShanghaiTech A/B, UCF-QNRF, and NWPU datasets demonstrate that our approach strikes a superior trade-off between parameter efficiency and counting accuracy and can be a good solution for crowd counting tasks in edge devices.

TCFormer: A 5M-Parameter Transformer with Density-Guided Aggregation for Weakly-Supervised Crowd Counting

TL;DR

This paper tackles the challenge of high annotation costs and heavy computation in crowd counting by introducing TCFormer, a tiny transformer framework trained with only image-level counts. It combines a compact TinyViT backbone, a Learnable Density-Weighted Averaging module for density-aware spatial aggregation, and a dual-head supervision scheme with a count regression head and a density-level classification head, all optimized with a multi-task loss. On ShanghaiTech A/B, UCF-QNRF, and NWPU, TCFormer achieves competitive or superior accuracy while maintaining a small footprint (5M parameters) and modest compute (about 1.18 GFLOPs), outperforming many weakly supervised baselines and rivaling heavier supervised models. The approach also demonstrates practical edge deployment potential, with low GPU power consumption (~59 W) and favorable energy per image, illustrating that density-aware aggregation plus global context can bridge the gap between weak supervision and deployment on resource-constrained devices.

Abstract

Crowd counting typically relies on labor-intensive point-level annotations and computationally intensive backbones, restricting its scalability and deployment in resource-constrained environments. To address these challenges, this paper proposes the TCFormer, a tiny, ultra-lightweight, weakly-supervised transformer-based crowd counting framework with only 5 million parameters that achieves competitive performance. Firstly, a powerful yet efficient vision transformer is adopted as the feature extractor, the global context-aware capabilities of which provides semantic meaningful crowd features with a minimal memory footprint. Secondly, to compensate for the lack of spatial supervision, we design a feature aggregation mechanism termed the Learnable Density-Weighted Averaging module. This module dynamically re-weights local tokens according to predicted density scores, enabling the network to adaptively modulate regional features based on their specific density characteristics without the need for additional annotations. Furthermore, this paper introduces a density-level classification loss, which discretizes crowd density into distinct grades, thereby regularizing the training process and enhancing the model's classification power across varying levels of crowd density. Therefore, although TCformer is trained under a weakly-supervised paradigm utilizing only image-level global counts, the joint optimization of count and density-level losses enables the framework to achieve high estimation accuracy. Extensive experiments on four benchmarks including ShanghaiTech A/B, UCF-QNRF, and NWPU datasets demonstrate that our approach strikes a superior trade-off between parameter efficiency and counting accuracy and can be a good solution for crowd counting tasks in edge devices.
Paper Structure (36 sections, 15 equations, 5 figures, 7 tables)

This paper contains 36 sections, 15 equations, 5 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Fully-Supervised Crowd Counting
  4. Weakly-Supervised Crowd Counting
  5. Lightweight Crowd Counting
  6. Methodology
  7. TinyViT Backbone
  8. Learnable Density-Weighted Averaging Module
  9. Multi-Task Prediction Heads
  10. Count Regression Head
  11. Density-Level Classification Head
  12. Overall Loss
  13. Experiments and Results
  14. Experimental Setup
  15. Datasets
  16. ...and 21 more sections

Figures (5)

  • Figure 1: The crowd counting framework: TCFormer.
  • Figure 2: The nMAE & nRMSE value of some models on benchmark datasets.
  • Figure 3: Comprehensive multi-view comparison of crowd-counting models. (a) FLOPs–Parameters plot showing computational cost. (b) Accuracy comparison using normalized MAE (nMAE) and normalized RMSE (nRMSE). (c) Joint efficiency–accuracy tradeoff, with FLOPs on the x-axis, nMAE on the y-axis, marker size proportional to model parameters, and marker color encoding nRMSE.
  • Figure 4: The power consumption curves of the models
  • Figure 5: Visualization of Intermediate MBConv Feature Map