Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generalized-Scale Object Counting with Gradual Query Aggregation

Jer Pelhan, Alan Lukezic, Matej Kristan

TL;DR

This paper tackles few-shot counting in imagery with large scale variation and dense object regions by introducing GECO2, a detection-based counter that learns scale-generalized dense queries through scale-specific encoders and cross-scale aggregation.GECO2 constructs high-resolution dense object queries at multiple scales, enabling accurate localization and counting without resorting to ad-hoc input upsampling or image tiling, and integrates SAM2 segmentation for refined masks.Through scale-aware prototypes, deformable attention, and lightweight fusion, GECO2 achieves superior counting and detection performance across FSCD147, FSCD-LVIS, and MCAC benchmarks while reducing memory usage and speeding up inference by about 3× relative to prior state-of-the-art methods.A comprehensive set of ablations demonstrates the contributions of multi-scale query construction, auxiliary supervision for small objects, and the efficiency of the cross-scale aggregation approach, underscoring its practical impact for real-world, densely populated scenes.

Abstract

Few-shot detection-based counters estimate the number of instances in the image specified only by a few test-time exemplars. A common approach to localize objects across multiple sizes is to merge backbone features of different resolutions. Furthermore, to enable small object detection in densely populated regions, the input image is commonly upsampled and tiling is applied to cope with the increased computational and memory requirements. Because of these ad-hoc solutions, existing counters struggle with images containing diverse-sized objects and densely populated regions of small objects. We propose GECO2, an end-to-end few-shot counting and detection method that explicitly addresses the object scale issues. A new dense query representation gradually aggregates exemplar-specific feature information across scales that leads to high-resolution dense queries that enable detection of large as well as small objects. GECO2 surpasses state-of-the-art few-shot counters in counting as well as detection accuracy by 10% while running 3x times faster at smaller GPU memory footprint.

Generalized-Scale Object Counting with Gradual Query Aggregation

TL;DR

This paper tackles few-shot counting in imagery with large scale variation and dense object regions by introducing GECO2, a detection-based counter that learns scale-generalized dense queries through scale-specific encoders and cross-scale aggregation.GECO2 constructs high-resolution dense object queries at multiple scales, enabling accurate localization and counting without resorting to ad-hoc input upsampling or image tiling, and integrates SAM2 segmentation for refined masks.Through scale-aware prototypes, deformable attention, and lightweight fusion, GECO2 achieves superior counting and detection performance across FSCD147, FSCD-LVIS, and MCAC benchmarks while reducing memory usage and speeding up inference by about 3× relative to prior state-of-the-art methods.A comprehensive set of ablations demonstrates the contributions of multi-scale query construction, auxiliary supervision for small objects, and the efficiency of the cross-scale aggregation approach, underscoring its practical impact for real-world, densely populated scenes.

Abstract

Few-shot detection-based counters estimate the number of instances in the image specified only by a few test-time exemplars. A common approach to localize objects across multiple sizes is to merge backbone features of different resolutions. Furthermore, to enable small object detection in densely populated regions, the input image is commonly upsampled and tiling is applied to cope with the increased computational and memory requirements. Because of these ad-hoc solutions, existing counters struggle with images containing diverse-sized objects and densely populated regions of small objects. We propose GECO2, an end-to-end few-shot counting and detection method that explicitly addresses the object scale issues. A new dense query representation gradually aggregates exemplar-specific feature information across scales that leads to high-resolution dense queries that enable detection of large as well as small objects. GECO2 surpasses state-of-the-art few-shot counters in counting as well as detection accuracy by 10% while running 3x times faster at smaller GPU memory footprint.

Paper Structure

This paper contains 15 sections, 6 equations, 3 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. GeCo2
  4. Scale-Specific Query Encoder
  5. Cross-Scale Query Aggregation
  6. Dense Query Decoder
  7. Training
  8. Experiments
  9. Implementation details
  10. Experimental protocols and benchmarks
  11. State-of-the-Art Comparison
  12. Comparison with scaling-heuristic-free sota
  13. Memory and inference speed
  14. Ablation Study
  15. Conclusion

Figures (3)

  • Figure 1: GeCo geco struggles with diverse object scales and CountGD countgd fails in dense scenes with small objects. GeCo2 overcomes these issues by the new scale-generalized dense queries and achieves superior accuracy, fast inference and a substantially lower memory footprint.
  • Figure 2: The architecture of the proposed generalized-scale detection-based object counter GeCo2.
  • Figure 3: GeCo2 and GeCo2$_{FP}$ with removed multi-scale query generation, using $\mathbf{C}_\text{2}$ and $\mathbf{C}_\text{3}$ for upsampling.