CoDi -- an exemplar-conditioned diffusion model for low-shot counting

TL;DR

CoDi tackles low-shot counting by marrying exemplar-conditioned diffusion in latent space with a diffusion-denoising backbone to produce precise object-location maps. The method introduces adaptive exemplar prototypes integrated into the UNet, plus a reference-less adaptation path, enabling robust counting from few to no exemplars. Empirical results on FSC147 and MCAC show strong gains over density-, detection-, and localization-based counters, with improved localization quality and competitive inference efficiency. This approach offers a practical, scalable solution for accurate counting and localization in crowded, small-object scenes, with potential extensions toward bounding-box prediction.

Abstract

Low-shot object counting addresses estimating the number of previously unobserved objects in an image using only few or no annotated test-time exemplars. A considerable challenge for modern low-shot counters are dense regions with small objects. While total counts in such situations are typically well addressed by density-based counters, their usefulness is limited by poor localization capabilities. This is better addressed by point-detection-based counters, which are based on query-based detectors. However, due to limited number of pre-trained queries, they underperform on images with very large numbers of objects, and resort to ad-hoc techniques like upsampling and tiling. We propose CoDi, the first latent diffusion-based low-shot counter that produces high-quality density maps on which object locations can be determined by non-maxima suppression. Our core contribution is the new exemplar-based conditioning module that extracts and adjusts the object prototypes to the intermediate layers of the denoising network, leading to accurate object location estimation. On FSC benchmark, CoDi outperforms state-of-the-art by 15% MAE, 13% MAE and 10% MAE in the few-shot, one-shot, and reference-less scenarios, respectively, and sets a new state-of-the-art on MCAC benchmark by outperforming the top method by 44% MAE. The code is available at https://github.com/gsustar/CoDi.

Figures (11)

• Figure 1: State-of-the-art detection-based counters (eg., CountGD amini2024countgd) struggle with small objects in dense regions, and often producing false positives. The proposed CoDi employs adaptive prototype extraction and diffusion-based location refinement, leading to accurate detections and counts.
• Figure 2: CoDi employs a latent conditioned diffusion model with adaptive prototype extraction and a modulated Denoising UNet to gradually refine object locations map $\tilde{\mathbf{L}}$.
• Figure 3: The exemplar-based conditioning module (EC) is applied in the first and last layer of the Denoising UNet.
• Figure 4: CoDi shows better discrimination between different categories and robustness to double detections and false negatives compared to the version with external conditioning (CoDi$_{\text{ex}}$). Yellow dots denote detections, while ERR is count error.
• Figure 5: Reducing the diffusion process to $T=2$ timesteps, yields inadequate density maps for peak counting (numbers indicate the count error after applying NMS). Increasing to $T=25$ timesteps, CoDi converges, producing high quality peaks.