High-Quality Proposal Encoding and Cascade Denoising for Imaginary Supervised Object Detection

TL;DR

This work tackles Imaginary Supervised Object Detection (ISOD) by elevating synthetic data quality and introducing architectural innovations to DETR-based detectors. It combines a high-quality synthetic data pipeline (FluxVOC/FluxCOCO) with HQP-guided query encoding that initializes queries from image-specific priors using SAM proposals and RoI-pooled features, and a cascade denoising scheme that progressively increases IoU thresholds across decoder layers to mitigate pseudo-label noise. The approach yields state-of-the-art results on VOC2007 mAP@0.5 after only 12 epochs and demonstrates strong cross-domain generalization, including competitive real-data performance, validating its universal applicability. These contributions offer a practical path to high-performance object detection when real labeled data are scarce, by tightly integrating data quality, query initialization, and robust training under noisy supervision.

Abstract

Object detection models demand large-scale annotated datasets, which are costly and labor-intensive to create. This motivated Imaginary Supervised Object Detection (ISOD), where models train on synthetic images and test on real images. However, existing methods face three limitations: (1) synthetic datasets suffer from simplistic prompts, poor image quality, and weak supervision; (2) DETR-based detectors, due to their random query initialization, struggle with slow convergence and overfitting to synthetic patterns, hindering real-world generalization; (3) uniform denoising pressure promotes model overfitting to pseudo-label noise. We propose Cascade HQP-DETR to address these limitations. First, we introduce a high-quality data pipeline using LLaMA-3, Flux, and Grounding DINO to generate the FluxVOC and FluxCOCO datasets, advancing ISOD from weak to full supervision. Second, our High-Quality Proposal guided query encoding initializes object queries with image-specific priors from SAM-generated proposals and RoI-pooled features, accelerating convergence while steering the model to learn transferable features instead of overfitting to synthetic patterns. Third, our cascade denoising algorithm dynamically adjusts training weights through progressively increasing IoU thresholds across decoder layers, guiding the model to learn robust boundaries from reliable visual cues rather than overfitting to noisy labels. Trained for just 12 epochs solely on FluxVOC, Cascade HQP-DETR achieves a SOTA 61.04\% mAP@0.5 on PASCAL VOC 2007, outperforming strong baselines, with its competitive real-data performance confirming the architecture's universal applicability.

Figures (8)

• Figure 1: Qualitative comparison of our FluxVOC (bottom) and the baseline ImaginaryNetimaginarynet (top). Our pipeline achieves superior results in prompt richness, visual fidelity, and annotation granularity.
• Figure 2: Overview of Cascade HQP-DETR. Backbone + Encoder (with Dense Fusion) provide features to a decoder processing two branches: the HQP branch initializes object queries with SAM proposals as anchor boxes and RoI-pooled features (via Neck) as content embeddings; the denoising branch initializes denoising queries with noisy GT boxes as anchors and their RoI-pooled features as content embeddings. HQP uses IoU-guided Hungarian matching and IoU-guided classification/regression losses. Denoising applies layer-wise increasing thresholds $\theta_l$, computing $w_{l,j}$ from the IoU between layer-$l$ reconstructed boxes and GT to dynamically modulate features. Inference removes denoising, with predictions optionally refined by SAM.
• Figure 3: Visual examples from our synthetically generated FluxVOC dataset, shown with their automated bounding box annotations. The collage is thematically organized by column to showcase the dataset's breadth. From left to right, the columns display: (1) various modes of transportation, (2) common indoor objects, (3) complex, multi-instance human-centric scenes, and (4) a diverse range of animals. This structure highlights our pipeline's ability to generate high-quality images across distinct domains, including challenging scenarios with small, dense, and occluded objects. (Best viewed in color and with zoom).
• Figure 4: First-layer matched queries for DAB-DETR dabdetr (red) vs. our Cascade HQP-DETR (green).Synthetic Domain (top two rows): On the FluxVOC training data, both models exhibit high initial IoU, demonstrating their ability to fit the synthetic distribution.Real Domain (bottom two rows): On the real VOC07 pascalvoc test set, our model's queries remain highly accurate, while the baseline's degrade significantly. This highlights our model's superior generalization across the synthetic-to-real domain gap.
• Figure 5: KDE of bounding box centers. ImaginaryNet (left): extreme center bias. FluxVOC (center): broad, multi-modal. Real VOC 2012 (right): natural, broad coverage.