CountFormer: A Transformer Framework for Learning Visual Repetition and Structure in Class-Agnostic Object Counting

TL;DR

CountFormer introduces a structure-aware, exemplar-free counting framework by replacing the encoder with the self-supervised DINOv2 foundation model and incorporating 2D positional embeddings to preserve geometry, then decoding into a density map for counting. Trained and evaluated on FSC-147, it achieves competitive zero-shot performance and shows superior capability for structurally complex or densely repetitive scenes, addressing miscounts caused by object parts and overlapping components. The approach demonstrates that foundation-model embeddings can imbue counting systems with spatial reasoning akin to human perception, moving toward truly general, exemplar-free counting. Limitations remain in extremely densely packed or uniform object scenarios, suggesting higher-resolution inputs or adaptive processing as future work with potential impact on broad, real-world counting tasks.

Abstract

Humans can effortlessly count diverse objects by perceiving visual repetition and structural relationships rather than relying on class identity. However, most existing counting models fail to replicate this ability; they often miscount when objects exhibit complex shapes, internal symmetry, or overlapping components. In this work, we introduce CountFormer, a transformer-based framework that learns to recognize repetition and structural coherence for class-agnostic object counting. Built upon the CounTR architecture, our model replaces its visual encoder with the self-supervised foundation model DINOv2, which produces richer and spatially consistent feature representations. We further incorporate positional embedding fusion to preserve geometric relationships before decoding these features into density maps through a lightweight convolutional decoder. Evaluated on the FSC-147 dataset, our model achieves performance comparable to current state-of-the-art methods while demonstrating superior accuracy on structurally intricate or densely packed scenes. Our findings indicate that integrating foundation models such as DINOv2 enables counting systems to approach human-like structural perception, advancing toward a truly general and exemplar-free counting paradigm.

Figures (6)

• Figure 1: Detailed overview of the proposed model architecture: Our approach utilizes a self-supervised visual encoder, specifically DINOv2, to process the query image. The extracted image features are then summed with the positional embeddings which serve as query vectors. Subsequently, we use a CNN-based decoder to decode these vectors and upscale them to generate the corresponding density map. Finally, the total object count is determined by integrating the values across the density map.
• Figure 2: For the given image (left) with a ground truth of 149 objects, our model produced a density map (right) that was summed to predict an object count of 149.
• Figure 3: Failure examples illustrating limitations of our model on images with large object counts and weak inter-object separation. Top row: test set; bottom row: validation set.
• Figure 4: Qualitative results of our model on FSC-147. Each sub-figure consists of the original image (left), CounTX's density map (middle), and our model's density map (right). The predicted counts from both models and ground truth (GT) counts have also been provided.
• Figure 5: Comparison on the “glasses” example from FSC-147. Ground truth (GT) count = 96; CounTX prediction = 185; our model’s prediction = 98. Left to right: input image, CounTX density map, and our model’s density map.