Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ProgRoCC: A Progressive Approach to Rough Crowd Counting

Shengqin Jiang, Linfei Li, Haokui Zhang, Qingshan Liu, Amin Beheshti, Jian Yang, Anton van den Hengel, Quan Z. Sheng, Yuankai Qi

TL;DR

ProgRoCC tackles crowd counting with rough labels, addressing annotation bottlenecks by using digit-wise counts in a CLIP-based framework. It introduces Progressive Estimation Learning to predict counts from high to low digits within a $0$--$999$ range, and a Visual-Language Matching Adapter to refine cross-modal features. Experiments on SHA, QNRF, and JHU++ show substantial gains over unsupervised and semi-supervised methods and competitive results against some fully supervised baselines, with notable cross-dataset transferability. The approach yields faster inference (about 30 CLIP text matches per image) and reduces labeling costs, enabling scalable deployment in diverse scenes.

Abstract

As the number of individuals in a crowd grows, enumeration-based techniques become increasingly infeasible and their estimates increasingly unreliable. We propose instead an estimation-based version of the problem: we label Rough Crowd Counting that delivers better accuracy on the basis of training data that is easier to acquire. Rough crowd counting requires only rough annotations of the number of targets in an image, instead of the more traditional, and far more expensive, per-target annotations. We propose an approach to the rough crowd counting problem based on CLIP, termed ProgRoCC. Specifically, we introduce a progressive estimation learning strategy that determines the object count through a coarse-to-fine approach. This approach delivers answers quickly, outperforms the state-of-the-art in semi- and weakly-supervised crowd counting. In addition, we design a vision-language matching adapter that optimizes key-value pairs by mining effective matches of two modalities to refine the visual features, thereby improving the final performance. Extensive experimental results on three widely adopted crowd counting datasets demonstrate the effectiveness of our method.

ProgRoCC: A Progressive Approach to Rough Crowd Counting

TL;DR

ProgRoCC tackles crowd counting with rough labels, addressing annotation bottlenecks by using digit-wise counts in a CLIP-based framework. It introduces Progressive Estimation Learning to predict counts from high to low digits within a -- range, and a Visual-Language Matching Adapter to refine cross-modal features. Experiments on SHA, QNRF, and JHU++ show substantial gains over unsupervised and semi-supervised methods and competitive results against some fully supervised baselines, with notable cross-dataset transferability. The approach yields faster inference (about 30 CLIP text matches per image) and reduces labeling costs, enabling scalable deployment in diverse scenes.

Abstract

As the number of individuals in a crowd grows, enumeration-based techniques become increasingly infeasible and their estimates increasingly unreliable. We propose instead an estimation-based version of the problem: we label Rough Crowd Counting that delivers better accuracy on the basis of training data that is easier to acquire. Rough crowd counting requires only rough annotations of the number of targets in an image, instead of the more traditional, and far more expensive, per-target annotations. We propose an approach to the rough crowd counting problem based on CLIP, termed ProgRoCC. Specifically, we introduce a progressive estimation learning strategy that determines the object count through a coarse-to-fine approach. This approach delivers answers quickly, outperforms the state-of-the-art in semi- and weakly-supervised crowd counting. In addition, we design a vision-language matching adapter that optimizes key-value pairs by mining effective matches of two modalities to refine the visual features, thereby improving the final performance. Extensive experimental results on three widely adopted crowd counting datasets demonstrate the effectiveness of our method.

Paper Structure

This paper contains 16 sections, 9 equations, 3 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Crowd Counting
  4. Contrastive Language-image Pre-training
  5. Method
  6. Progressive Estimation Learning
  7. Visual-Language Matching Adapter
  8. Experiment
  9. Datasets and Evaluation Metrics
  10. Implementation Details
  11. Comparison with the State-of-the-Art Methods
  12. Ablation Study
  13. Effectiveness of key components of ProgRoCC.
  14. Effect of rough labels.
  15. Cross-dataset Validation
  16. ...and 1 more sections

Figures (3)

  • Figure 1: Our solution to rough crowd counting. Compared to traditional counting methods that require accurate annotations, our method achieves robust crowd counting using simpler approximate estimates of object numbers as labels. It captures the number of people through a progressive estimation learning strategy.
  • Figure 2: Overview of our proposed ProgRoCC. We freeze the text encoder in pretrained CLIP and finetune the image encoder by a progressive estimation learning (PEL) strategy (Sec. \ref{['sec:PEL']}). Then, a visual-language matching adapter (VlMA, Sec. \ref{['sec:VLMA']}) is introduced to improve the image features for more reliable visual-to-text matching. The green straight arrows and the black straight arrows denote the stages of training PEL and VlMA sequentially.
  • Figure 3: Examples of predicted results on SHA.